BigLots: lol! I laughed at that last statement, because I think many people probably feel the same way. Anyway, depending on where you live, I'd say that one of the best and inexpensive places may be Biglots. I shop there myself because they have a wide variety of things, including those mentioned in your question. Good quality stuff, really nice decorative things and decent prices to hook up a room or something in a snap.
Freds: This place is limited it seems, to mainly the southern regions of the country, but if you have a Freds nearby, then you can't go wrong. They have many things that come from higher end department stores that are no longer marketable for whatever reason. This means that the prices are slashed tremendously. I will say this though, stay away from the apparel. I find that their clothes are rather cheap and flimsy looking, while not being much different in price than any other store.
Dollar stores: These are really good places too, but every dollar store isn't the same, especially the ones that sell things way over "a dollar" lol. Out of the many that are out there, I'd have to say that FamilyDollar is at the top of my list. They have a good variety of stuff and their food items are particularly cheap yet of good quality brands. Dollar mart, if you have one, is also a great place. You'll have to visit them frequently to catch that deal, but when you catch it...it's one heck of a deal. I've seen Huggies pampers in there (32 count) for only $5, and brand new clothing for the little ones as cheap as 50 cents.
Hope these help. Didn't mean to turn this answer into an article in itself by the way...lol