Marriage is (man-made) ceremony. If one believes in the bible then they are aware that there was no minster that married Adam & Eve and yet they were considered husband and wife.
Historically during the slave trade slaves simply "jumped the broom" and considered themselves married. They weren't legally allowed to have wedding licenses. They considered themselves married.
In the United States, common law marriage has been in existence since the horse and buggy days of 1877.
A common law marriage is one in which the couple, usually a man and woman, lives together for a period of time and holds themselves out to friends, family and the community as "being married," but never go through a formal ceremony or get a marriage license.
The following states fully recognize common law marriage: Alabama, Colorado, District of Columbia, Iowa, Kansas, Montana, Rhode Island, South Carolina, Texas (calls it "informal marriage") and Utah. If you live in one of these states and are considering such an arrangement you should look into your state's requirements.
Note: Lots of people get married in their local courthouse by the justice of the peace. Marriage is about (legality) and not religion in the eyes of the government.