Christin.....My favorite type of question....right out of "Behavioral Science," and right up my alley........The kind of person you describe as a "compulsive liar," (which is an accurate term to use), does, in fact, boil down to a Pathology (disease) aka "Pathological Liar".....recent high profile examples: Casey Anthony and Jody Arias......both severely full-blown, Pathological Liars with Borderline Personality Disorder. These two disorders, more often than not, accompany one another, as well as probable stark narcissism.
Perhaps, unbeknownst to you, the summary of your question, has many answers to your own question....For instance, you use the word, "irrational," and the phrase, "invent all sorts of drama".....Classic examples. The other behavioral word that stands out blatantly with these individuals is "Exaggeration." Fantastic-like explanations, using every form of superlative....and unnecessary added BLING.
WHY? you ask, do people feel COMPELLED to lie?.....when.....
Again, the answer is within your question. Lying IS precisely, a Compulsion, where these sad creatures are concerned. Knee-jerk reaction, of which they have little or no conscious control. (key word being, "conscious.") The Pathological Liar is on constant auto-pilot.
As I write this, I am well aware I should simply accept that there is a HUB here, I must write. Good for you, Christin! You have inspired creation.
I dare not leave you hanging, though. Suffice it to say that the genesis of this mental issue, can most often and in large part, be traced back to fairly early childhood. 1. The overly protected and "adored" child 2. The child who in his/her own mind, whether justifiable or not, firmly believe he/she is the LEAST favorite..(black sheep/outsider) within his/her group of siblings. 3. The infamous double-edged sword.... a child placed on an unrealistic pedestal, yet continually criticized and scorned. The latter occurs most often when one or both parents insist upon living vicariously, through their children. These parents attempt to "create" the person they had wanted to be, but never achieved.....OR...did achieve and wish to create a sort of "Mini-Me"...
You get the gist, I'm sure. Imagine a decade or more of living in constant & dreaded FEAR of the TRUTH. They learn to lie....quickly and well, in order to maintain favor........Superb Question! I hope you receive numerous responses.