I have tried to find the answer to this question before because I was curious about it myself. The best study I have found so far is the one made by the Foreign Service Institute of the US Department of State a few years back. They were trying to measure how long native English speakers who already knew other foreign languages would take to reach proficiency in reading and speaking in several foreign languages.
Of all the languages they have analyzed, Japanese was the hardest to learn. Other languages classified as "exceptionally difficult for native English speakers" included Arabic, Cantonese, Korean and Mandarin. The easiest were, not very surprisingly, Afrikaans, Danish, Dutch, Norwegian, Swedish and several Romance languages.
Unfortunately, the study didn't include as many languages as I would have liked - only 63 - but that's a fair amount nonetheless. The results for all the languages studied are on the link below (sorry, I had to pull it using the WayBack Machine).
I hope that satisfies your curiosity :)