I think reliability and good customer and technical service are the most important "features". Of course if speed is given.
HostGator is one of the best hosting companies out there. I have been with them for over a year now and I just signed my friend up and I used coupon code: HOST994HGH for him and saved him close to $10. I have been with at least 5 other companies and HostGator is really one of the best.
Other good companies are : KnownHost and PowerVPS for larger accounts.