Get AWAI's CD. They offer step-by-step instuctions on how to start your own creative business.
Some of the first things you need to do is choose a name and file LLC or other corporate papers at the state corporation commission, get a business licence, and bank account, and possibly an EIN or TIN number from the IRS. Hopefully you have the software you need to get started (for design) and possibly QuickBooks or other accounting software.
Most people choose a generic name like Jessica Jones Graphic Design, because there's less trademark issues to worry about. You can hire a lawyer to file LLC or Inc papers for you, but it's not necessary. You can also trademark a unique name, but I would not spend a ton a money right away unless you can afford to. If you already have the graphics software you need, you should be able to get going for a few hundred dollars. If you don't have graphic design software yet, that could cost a lot more.
Create your own website or blog and put your portfolio on it. Make contacts with family and friends, advertise, join the Chamber of Commerce to network to get the business of businesses.