Everything I make I do by hand, so there's no need for expensive dye cut machines or things like that. A great way to start is to look up some layout templates, get an idea of what you want (what are the pictures you want to use, what's the theme?) to create so that you aren't going to the store meandering the aisles with no clue! Instead of buying paper packs that can be pricey, buy single sheets that coordinate with each other. If you have the money to spare, paper packs are a better deal but require you to spend more initially.
Buy an exacto knife! It's been my best friend! If i see an embellishment I like online, I print out a picture of it, cut a template and cut it out with the exacto knife. This can save you hundreds of dollars a year on just embellishments! Besides, you can customize it with the paper that matches your theme.
Know where to shop! Scrapbooking can be a really expensive hobby! If there's a Hobby Lobby near you, they are constantly have 50% off sales. Craft stores have better sales more often than specialty stores, such as Archivers.
Buy off-season! I know you specifically said "right away" in your question, but you can save tons of money buying clearance. I always buying seasonal and holiday embellishments/papers once they are on clearance and save it for next year.
I hope I wasn't too long-winded and that this helps you! Let me know if you have anything else you want me to address!