Ken, make sure to take your time. Don't take the previous owner's word for anything. Horses are herd animals that have a well-developed hierarchy. Just because the horse respects the current owner as a dominant member of the herd doesn't mean they are going to respect you - especially as a first timer. Ride the horse, have a knowledgeable friend ride the horse, and then ride the horse some more.
My wife and I (an absolute beginner at almost 50) bought two horses from a family that needed to downsize. When we went to look at them, the owner jumped up on my horse bareback and rode him in the just a halter. For me, once we got him home in the new setting, he was skittish and jumpy. Four months, several nervous trail rides and three broken ribs later (mine), we sold him to a more knowledgeable, experienced rider.
I now have a calm, collected Missouri Foxtrotter. I wrote about the breed at https://hubpages.com/animals/What-is-a-Missouri-Fo... First timers want a quiet, dependable, bomb-proof horse. If you see ANYTHING on the test rides that concerns you, don't try to justify - just WALK AWAY!
My wife is currently looking for another horse for herself, since her mare was moody and unfriendly. She started riding when she was eight and continued until college. Now that she is getting back into it, she is realizing that she puts way too much emphasis on how a horse looks, rather than how it acts. So she always takes a knowledgeable friend along to be the voice of reason. It makes for a long and trying buying process, but it's better than having a horse you don't want on your hands.