"my worry is my man told me that I cannot come to his house while she is there homeless ....he doesn't mind if I visit during the day."
A couple of months ago you asked a question related to this situation before his ex arrived. Apparently you decided to stick with him.
If my significant other had her ex staying with her and this man was running around the house naked trying to seduce her I'd move on.
Imagine if you were not in this situation 5 years ago and your sister or a best friend came to you and told you that her boyfriend's ex and their baby has moved in with him. She's running around the house naked trying to seduce him but he insists nothing is going to happen between them. What would you advise her to do?
Most people don't want their mate meeting exes socially for lunch or dinner and yet you're in a situation where your boyfriend is (living with) his ex!
Only you can decide what is "right" for you but keep in mind there are over 7 Billion people on this planet! Surly you can find a man who doesn't share a house with a naked ex who wants to get him back!
This scenario is equivalent to driving your car into a bad neighborhood, rolling down the windows, leaving it running with your key in the ignition and praying each night that no one steals it!
My guess if it were the other way around and you had an ex-boyfriend staying with you who was constantly hitting on you and running around the house naked and you told your boyfriend he couldn't come over....etc He'd drop kick you in a heartbeat!
There are only two ways to experience joy and peace of mind in relationships: We either get what we want or we learn to be happy with what we have. Accept things as (is) or move on.
The choice is up to you!