I didn't go to my prom because I grew up in a very restrictive home and wasn't allowed to date or go to many "afterschool" activities and events. It was awkward because at that time and where I lived, it was very much an expected thing that everyone did. At the time you also couldn't go solo to our school's prom. (This was in the late 1980s in a very rural community.)
I even remember today how awkward it was when basically everyone in my class got dismissed early from school so they could go get ready, get their hair done up and prepare for the big night. The teachers didn't even seem to know what to do with the few of us who weren't going, so we ended up being stuck in the school library for the afternoon until regular dismissal time. Not a great memory! But not anything that ruined my life, that's for sure. It did make me "weird" at the time, but I don't think today it's nearly so big of an issue.