First, answer this question...
Would you set aside your relationship with your son to make a man that you've been dating for six months feel more secure and "included" in your life? (Especially if your son was only four!)
If your answer to that question is YES, then you aren't much of a parent, you don't "understand" the single parent dynamic, you feel jealous of a four-year old child and resent the fact that her father loves her and wants to make sure she is emotionally secure, and you appear to be extremely shallow and self-absorbed.
Just because you are "exclusively" dating someone doesn't mean that you are "the one" and will be in that man's life for any substantial length of time or that you have some sort of "status" in his life beyond the bedroom door. Especially if you aren't included in "family" functions.
Also, if the little girl's mother is a vindictive sort of parent, he may keep you at arm's length when his daughter is visiting him to keep "reports" from being delivered back to mom and him losing his weekend visitation altogether. Or he may not want to explain to his daughter why there's another woman in his life in a similar emotional/intimate situation as her mother was...at least not at the six-month point.
Six months really isn't any time at all for a single parent of a very small child to start introducing his newest flame to his little princess as her future "auntie" or potential new mommy.
You should rejoice in the fact that he man wants to spend time exclusively with his daughter and not "share" that time with you....which would certainly take away from the attention he would be giving to his little girl. This shows that he is trying to be the best possible parent he can under the circumstances.
If you love him as you claim and you are confident that he loves you, be glad that you have a place in his life at all. Things will work themselves out in time...for better or worse.