They are made of different materials, this can effect how heavy they are by quite large amounts (realatively).
Some people prefer heavier, som lighter. The distribution of the weight can be a factor too. Some people prefer the head of the racket to be heavier, and some prefer the weight distribution between the head and the grip to be pretty even (like me).
A cheap racket might not be strung so well. But you can always get it restrung at some point, and will may have to anyway if you break a string! Top players will often get them restrung automatically, as strings can slacken or suffer if left in a shop etc for ages.
All the other factors are pretty much a question of taste, such as the size and type of grip. Generally you will get better quality racket, the more you pay. However, if you are unsure about the weighting, I wouldn't pay out a lot until you've have chance to try different rackets.
Famous brands include: Yonex, Wilson, Carlton, Head, Kason. I have played with Yonex, Carlton and Head at various times, myself (although I've switched to tennis and stopped badminton since I moved from the UK to Florida!).
I wouldn't go below about $25-30 if you are serious about learning. But wouldn't spend too much either, until you know what you like!