For me, it's the packing. Everything is in an uproar and I can't rely on where things are anymore. Unpacking and setting up house is actually much more fun for me. I've moved many, many times and enjoy the relocation part. It's always an adventure. But packing? Horrid!
Relocating is an art form. I've moved quite a lot and I think I have it down to a science. Packing isn't so bad...just give half of what you own, away to friends and your moving expenses are cut down. Self contained moving PODS make the job much easier now and self storage reduces the rush to move in. The most stressful part id finding the next place to live. Not knowing how the neighbors are going to receive me or whether I will have any problem neighbors.
Long staying at one place conditions your mind into certain fixed habits. Moving means change and change means embracing some unknowns. It is the fear of unknown that makes you apprehensive about the new place, new surroundings. But you settle down in few days or weeks and no more stress.