mice themselves are very clean, however they spread germs and leave fecal waste all over the place. That being said, you can go to the hardware store, buy small wooden mouse traps, set them with a bit of candy bar such as snickers or kitkat and place them in warm dark spaces like behind stove or fridge and soon you will have no more mouse.
The mouse is not dangerous to your health unless it bites you, or you happen to consume its waste (which I why I advise you clear out your pantry and search through your food for any droppings, as soon as possible). To find the mouse again, you can try to lure it out by leaving peanut butter on a trap, or you have to be vigilant and follow the trail of evidence it shall leave behind. I hope this helps