Tough question. Sylvia - I agree with edhan's answer. But, if you don't feel safe with them driving (and it sounds like you really don't) then yeah, you should definitely refuse to get in the car unless they let you drive. Because it is not just their safety they are putting at risk, but yours... ultimately, no matter who the person driving is, you shouldn't allow yourself to be in the car if you don't feel safe. You owe it to yourself and your spouse (if you have one) or future spouse & kids to keep healthy and safe. Could you just insist they get into YOUR car and YOU drive them around when you visit? Your dad will be more likely to be OK handing over the reins if it's your car. If you go by airplane to get to their place, you could even consider renting a car while you're there "to give me a bit more independence" if you need to put it politely to them. Then always give them rides to where they need to go - drive them in your rented car!
The other issue is them driving in general. If there have been near misses or if your dad's eye surgery has definitely decreased his driving ability, you need to point out (in a nice way) that it's not just himself and his wife and yourself that could be at risk, but other drivers and pedestrians on the road. One woman I knew of was very resistant to stopping driving. Her ability was terrible (she did have a disease which was causing her driving to get worse and worse). Finally her daughter told her it was time for her to stop driving. She didn't agree... until the daughter said "You remember that near miss you had? The person in the other car could easily have been killed. Well, imagine that in all of the other cars around you on the road, there is your granddaughter, your son, your sister. Do you still want to drive?" The woman said at that point, she realized she needed to stop driving.