I'm sure the answers above are more enlightening than mine, but I'd say the difference is that cakes are cakey, and muffins are muffiny.
It doesn't matter what shape they are (although to be a cupcake a cake has to be little and made in in cupcake shape). My question is, "What makes some things, like banana-nut bread and date-nut bread, "bread"? In my book, they're cake.
(And where does pound cake fit into the mix? - or should I say, flour or batter?)