Huge congrats for you to be thinking about the costs before getting a pet. The costs will vary GREATLY depending on what type of pet you get. A pet such as a horse, obviously, would be exceedingly expensive. Dogs and cats can be very pricey. Pocket pets, such as gerbils and hamsters, are going to be much less expensive.
Dogs are my specialty, so I'll focus on that. When I look at getting a new dog, I'm not looking at "annual" cost as I know full well that an unexpected health emergency can pop up. These can cost well into the thousands of dollars. I always want to know that I either have a savings account set aside for these potential emergencies OR I have pet health insurance. If I didn't have extra money set aside, I would DEFINITELY have pet health insurance. This way, no matter what, my pets health needs are taken care of.
Looking only at food, shots, spay/neuter, etc. is, IMO, being very short-sighted (and I can tell you are not doing that at all). I believe, like you, that taking on a pet involves more than just "monthly" expenses.
That being said, "normal" expenses would include things like collars, leashes, food, bowls, adequate fencing, grooming supplies, annual shots and vet visit, heartworm meds (VERY expensive and a MUST have), flea and tick products, bedding, crate, etc.
Extra things that I deem just as important include training classes, potential food supplements, training equipment and supplies, etc. If you really enjoy training and want to compete in a dog sport, then the costs really add up. :)
Other things that can develop that will cost money include behavioral issues. Behavioral issues can occur in properly raised dogs, so getting a puppy won't eliminate this potential problem. Extra money for training to ease or eliminate the behavioral issues also will need to be considered. An excellent trainer or true behaviorist (with a Masters degree) will cost upward of $100 an hour for in-home visits in many metro areas.
I have known dog after dog who have suffered because people have decided to get a dog because they "had their heart set on getting a pet." You, obviously, are not one of these people. These poor dogs wound up suffering needlessly or being euthanized because a simple medical treatment could not be afforded. Just taking care of heartworms or tick borne illnesses, which are on the rise, costs hundreds upon hundreds of dollars.
When it's time to get your pet, look into pet health insurance. :)