Yes you can!
By law in the USA former employers can give complimentary references and expound on them, but if the references are negative, they can only divulge your title, start and end dates, salary, and would re-hire or not answer.
This does not apply if involved in any legal action, in a contested unemployment inquiry/action, insurance litigation.
They run the risk of legal action on the grounds of libel/defamation if derogatory information is written and or slander if the information is spoken. This can be compounded if an intent of malice can be ascertained.
The first amendment has no bearing in these instances.