First of all, it should be noted that 75-80% of kittens have worms! The roundworm is pretty much invisible to the eye, so you won't see them in the litterbox. (If you do see worms in the stool, they are probably tapeworms and look like pieces of rice.)
So, if your ktten has a rounded and slightly protruding belly, you can pretty much be sure he's got worms. You can take a stool sample to the vet and they'll know in a day or two. The remedy are pills, 1/2 taken 3 times over 6 weeks. Mine cost $20 or so, a small price for a healthy kitten!