I don't think it's that. You don't really question 'to' someone. If you said, (no one to whom I spoke), then that would be better. I'm not sure how grammatically correct it is, but it's understandable, that's what counts.
You can help the HubPages community highlight top quality content by ranking this answer up or down.