When I was a young man I found this very difficult to say. I doubt I said it more than once or twice on the lower side of thirty or so. This would have been caused, I expect, by my Anglo Saxon upbringing in dear old England where shows of emotion in just about any instance were frowned upon. I believe this situation was even worse in some of the Germanic and Nordic countries.
However, as the years went by it was "I love you" in letters and cards and, eventually in actual spoken words to my darling wife. Once said only one time, it is easier the second time, then the third, and so on. But I've never let it become a meaningless habit, despite the fact I often call my wife, "Love." But then again, so do barmaids, and friendly shop keepers. Love is a common place term in Australia and doesn't really mean much at all.
Have I regretted not saying it enough? I think, 'Yes." I could have said this to my children as they grew up. But then again, as previously, stated, it isn't easy to get away from the way oneself was raised. I do say I love my children - all grown up now - now and again. But even after all of these years feel rather self-conscious about it.