Swimming is one of the best exercises, most certainly. It uses all your parts. So does yoga. However, are you after "in shape" or all over health?
Yoga has the added benefits of spiritual and mental health along with physical fitness and tightening and limbering. It's also stretching all in one :)
However, if you are looking for bodily health exercises, you want a combination of swimming, yoga, running, jumping - whatever. Whatever you do, you want something that gets your heart going faster than its normal speed.
While running isn't the most efficient exercise, it does something that yoga and swimming cant, which is strengthens your bones. The jolt of landing between each step stimulates your body to move more calcium to your bones.