This goes back to a pre-socratic debate over whether change was the fundamental property of the universe. Democritus brilliantly reconciled the opposing views by agreeing that they both were right and that there existed something that never changed, but the ever changing world was merely a reconfiguration of these unchanging things. He called them Atoms, thus atomic theory was reached with none of our modern science.
So in answer to your question I will ask another:
"Are you the matter that comprises you, or a specific range of configurations of that matter?"
Even if your matter is never created or destroyed, whatever 'you' are will certainly cease to exist once that matter has changed significantly enough to no longer represent whatever it is that you are.