It is based upon work ethic in addition to moral issues. It is assumed that if a person goes to bed & rises early, h/she is very industrious & diligent. There is the adage that the early bird gets the worm. This translates into people who rise early are not lazy & possess an admirable work ethic. People who go to bed early & rise early are viewed as above board, honest, & trustworthy morally as well in correlation to their work ethic.
On the contrary, people who go to bed & rise late have no work ethic. They are considered the partiers who would rather have a good time than to do an honest day's labor. Also, going to bed late & rising late have more sinister connotations. The nighttime is oftentimes associated with negativity, being sinister, & even evil. People who go to bed late & rise late are seen as up to no good as it is considered that the hub of human activity is in the daylight hours, Sleeping late means being active at night & people who are out at night are considered to be suspect as there is little or no activity during nighttime hours.