If you dont understand algebra 1 and 2 that you should of taken in High school than yes it is difficult, but if you are confident in those subjects, its basically the same thing. Also the teacher is a big factor
College algebra classes don't cover anything new as all the material you should have learned in high school, unless you were a remedial student. What could make it more difficult is homework and test questions that ask you to apply the math concepts to real-world problems.
In practice, most students find it manageable, usually an easy B if not an easy A.