It depends on whether I'm with somebody else or not, and it also depends on how long I want to keep keep the drink going if I'm not going to be with someone else. Also, convenience of transporting it can be factors.
Generally, I'd go with the 12 oz, which is kind of a standard size for hot coffee, and which means that if I'm with someone they'll most likely be getting that size too. That means, too, that one person isn't waiting for the other person to finish some giant cup of coffee. Also, hot coffee or tea either stay hot (which means having to worry more about spilling a hot drink when walking or driving); or else huge drinks get cold (and aren't as good when they do).
If the cold drink is iced tea I tend to prefer 16 oz. 12 oz is good for other drinks that have a lot of ice in them. Other than that, I lean toward 8 oz (and in some cases, I'd rather have a 6 oz cup) for "heavier" and/or carbonated beverages. I pretty much only drink iced tea outside when 'm hot, which is why I like the giant cup with lots of ice. I get other cold drinks (orange juice, soda maybe) for meal-related reasons and don't particularly want more than 8 oz at the most (and if that).
Coffee and tea are the only hot drinks I get, but if I were going to be alone and wanted to have the coffee or tea with me as I did something like work, then I'd go for, maybe, 20 oz. The bad thing about that, though, is sometimes the cup is a little riskier/difficult to handle. Also, there's more to spill (on, say, one's keyboard).
Although 12 oz is my standard for hot, and 16 is good for iced tea; I pretty much base my choices on "depending on the situation" and "depending on why I'm getting a drink in the first place".