There is nothing wrong with being single in my opinion.
Awhile back I asked the question: What does "single" really mean?
1. Some people take it to mean you are unmarried. (This is the "legal definition" on forms and applications)
2. Others took it another step and said it means you are not in a "exclusive relationship".
3. Still others said it means you are not dating anyone period!
Technically all 3 could be considered "single" depending on one's point of view. Being a male I can't say we experience very much pressure to get married or have children outside of possibly parents or maybe a woman who has been dating a guy for a couple of years.
As for casual sex some people view it as having sex with people you care nothing about. Others see having sex under the scenario of "friends with benefits" as being different than casual sex. In fact it is not uncommon for some seniors to (get together) with a "friend" from out of town a couple of times a year over the course of many years and they don't consider it to be "casual sex".
I've also known women in particular who choose to have sex with exes when they are in between relationships. Psychologically they feel that as long as they are not increasing the number of people they've had sex with and this person is not "someone off the street" it's not really "casual sex".
When it's all said and done no one should be judging anyone's lifestyle choices. Life is a (personal) journey. Each of us is entitled to live life on our own terms. If you're happy that's all that matters!