It might no be that you are under-estimating the time needed to travel a familiar route. What is probably happening is you remember the optimum time when everything went your way and there were no delays. Therefore, you remember the shortest time that you were able to complete that route.
When reality sets in there are many variables that can affect that travel time. You go back to the house to grab something you forgot (maybe more than once) and get away a few minutes later. There's traffic on the road.....or construction work that slows you down. When you reach your destination you have to cruise the parking lot to find a parking space and then the time it takes to walk into the building.
If you encounter all of the variables that I just listed and each adds two minutes to your travel time that would add 10 minutes to the time needed to reach your destination. And if there are more than one person involved in the trip the variables can escalate rapidly. That's why your Mom always told you to try and arrive 15 minutes before your appointment.