Some people are definitely harder to connect to than others. It may be because of bad experiences in the past meaning that people are more wary to 100% trust others.
In my own personal experience I seem to get on better with girls than guys and probably have 65% girl friends and the rest being male. However I probably have maybe 2-3 friends who I would say I can be completely 100% myself around with no judgment at all and trust them completely. So yes I wouldn't say I completely connect with all my friends.
Guys can definitely be good friends with girls without there being ulterior motives so nobody should be wary of you specifically for your gender...once they get to actually know you they will be more comfortable I'm sure.