They're both concerned with things that aren't 'real', so it's somewhat understandable that they tend to be lumped together.
Science fiction tends to be based on of what the future may be like, or how things could have been different if things had gone a different way. It's fiction with a basis in science, to put it simply. There's an implied plausibility in it - like, 'this could really happen'.
Fantasy is pure imagination. Fantasy elements of fiction don't need any basis in reality, or science. Magic, for example - magic is one of the stables of fantasy. Many authors put thought into internally consistent rules for how magic works, though most aren't overly concerned about justifying its existence. Unless they're doing something creative with a mix of sci fi and fantasy.
Beyond that, there's no real difference. The two genres can, and have, overlapped with each other. And, there are so many subgenres of each that trying to sort it all out can become a confusing mess.