Always improving is really important for me, and every artist I know. I don't think I've ever finished anything where I thought, "There's no way this could be better!" and yet, I learn something from every piece I create, so I have accomplished something.
Being educated in how to critique oneself is really handy for the confidence thing, because really, what is bad art? How do you know if what you are doing is good? Having an education in art is really helpful when you begin to question these aspects, and for the non-professionals, I would recommend just joining a group of artists that do have this knowledge, or taking a class or two.
Aside from that, I think many people have a very simple insecurity in expressing themselves so freely on paper, as though if someone looks at your art, they are looking at your soul. And what if they do not like what they see? Well, do you like what you see? Just another reason to always improve. ;)