It depends on a few things: culture, child's sociability, family's access to resources and parents' knowledge of what the child can already do well.
Each child develops socially, cognitively and physically a little different. If you thrust them into an environment that is foreign or overly stressful for them, they will not flourish or benefit from the activity. It's good to let them sample lots of different things if possible and see what they come back enthusiastic about. I would recommend starting before the child begins school as a way to ease them into the concept of structured days that make up classroom life and perhaps to help them socially.
As a parent, as long as you are not pushing the child into an activity, I'd say enroll them in whatever and however many you want. Extracurriculars are a HUGE part of college admittance these days, so the child's enrollment in various activities will set them up for success. Even if they quit the ones you picked, they may then choose their own because they will have learned a *lifestyle* that includes organized social activities, which they will likely continue to pursue throughout life. And in my opinion, that's a good thing.