You might not remember certain things that you learned earlier in school. I don't really think that's too important though. You also might not fit in with the younger crowd but that's also not important.
I think it has it's advantages for sure. You typically care more about your grades and doing well so you most likely will do better than you would if you were younger. The older students usually are the better ones. If you can do school without getting in to debt, all the better.