My response depends on a few factors: who is asking? what's our relationship? why am I saying no? what are the stakes?
For example, if a friend asked me to drive her to Montana (I live in Alabama) this weekend, I would say no without hesitation. Why did I say no? I'm busy this weekend, and gas is expensive. Would I say yes to such an absurd question any other time? Maybe. Especially if tequila is involved.
Example two...my boss asks me to work all weekend without compensation on a big grant (spoiler alert: this actually happened). I said yes. Why did I say yes? I need my job. If the grant got funded, by the grace of God, maybe I would get a bonus. Did I mention I was afraid to get fired for insubordination despite the absurdity of the request?
I'm definitely not afraid to say no, but I think it's most important to measure other factors (mentioned above) before jumping to an answer. If you need time to think about it, don't be shy about saying, "Let me think about it." It's your life. I could have just as easily said no to my boss, risked being sacked, and dealt with the consequences. Tactically, that wasn't a good move at that time, so I said 'yes.'