I can only speak for myself and not all atheists, but I believe in a type of energy. I call it the primal energy and I believe it is what travels throughout the universe and binds all atoms and molecules together. It's something like gravity, electricity, kinetic force or other physical property, we are just not yet able to see or analyze it. We are getting closer by discovering the Higgs-Boson particle, but there is still more to be found.
The primal energy or force, if you prefer, is what animates us as sentient beings. When our bodies die, they convert to this energy again.
So, as an atheist, I do not believe in a "soul" with my unique personality. I believe that mass (atomic structure) x energy (speed) = infinity. It's just that simple.
My atoms exist and have energy, therefore the essence of "me" is infinite.