First off let me say that I've been in the full-service restaurant business as busser, server, cook, and management for eight years now and as such I am normally very biased when it comes to tipping and restaurants in general. Buffets are a little different than full-service in my opinion, however.
I normally leave a $5 on the table in any situation that does not involve my children. I never allow my kids to make a huge mess, but if there is any mess at all I normally bump it to $8.
In most normal full-service restaurants the servers are making $2.13 or slightly more with the expectation that tips will on average push them over the minimum wage each hour. Typically a server will have 3-5 tables at a time and might only have 10-15 tables in one night, but there is a high degree of interaction with each table and the server will usually have a great impact on your dining experience. In this case tipping becomes more personal and typically we use percentages (15-20% for normal/good service) to reflect that level of personalization.
In buffet restaurants the servers make the same hourly wage, but in their case they might be taking care of 10 tables at a time and see 30-40 tables in one night! I personally feel that as the level of interaction with the table goes down it's less important to tip based on percentages. At that point for me it becomes a $5 transaction. They perform some quick and efficient tasks, exchange pleasantries, and get five bucks. Over the course of the night they are still making decent if not good money!
Keep in mind though that in a lot of Asian buffets and Mexican restaurants that there might be a lot of tip pooling going on and the money you are leaving on the table might not necessarily all end up in that particular server's pocket!