Not at all. My family has always done it as a traditional treat. Once every couple of months or so, someone will declare a "Breakfast At Night" dinner, and we will plan the menu during the day.
Sometimes we may just do pancakes, eggs, and sausage; other times it may be the full spread of eggs, ham, red-eye gravy, biscuits, bacon, pancakes and sometimes waffles.
For a lighter, quicker version: biscuits and sausage gravy with eggs.
Personally, I hate eating breakfast food in the morning. Smelling bacon or sausage first thing when I wake up has always been very repulsive to me, but I enjoy smelling them cook in the evening. So, breakfast at night is one of my favorite meals.