I am a mother of five, and with my oldest son, I allowed him to sleep in my room until he was 4 and this was a mistake. I know how hard it can be , especially once they are used to having you there and you them. Sometime I would be too tired to take him to his room , so I would give in and let him sleep in my room.
I am sure you know why this is not good for you , or your child. You do not get the rest you need, it creates a lack of intimacy with your partner.For the child it can create a unhealthy attachment, that you will see long after taking him out of the your room.
I know it is hard because as parents we too get attached to our kids. But your child later in life will need to have a inner sense of security , and if there are not able to feel safe when they are alone then , this will cause problems.
You can do this , it will take 7 days of not giving in at all. Give your child a night light and every time they get up put them back in bed.I know that it will be frustrating and make you tired, but after you stay strong you will see that this will never be a problem again, and your son will know you love him, so do not allow yourself to feel guilty you are doing what is right.