We use a Britta filter. I don't know how good it actually is, but I haven't noticed any impurities in the water, and it certainly tastes cleaner than regular tap water.
The filters don't cost a lot, and they last about three months or so, depending on how much use they get.
I like our Britta, so you can give that a try. I used to use the kind you screw onto the faucet, but we got a new sink which wasn't compatible and switched to a pitcher. There are other brands out there, but I can't say which is better.