You may not be able to refinance, but some mortgage companies may let you modify your mortgage. I have a mortgage with Navy Federal Credit Union in Vienna, VA. When the interest rates went down, I wanted to change my mortgage interest rate. Since I did not want to change the length of my loan or do a cash out refinance, they allowed me to modify the interest I was charged. All I had to do was sign a paper. There was no fee. I didn't receive the absolute lowest rate in the market, but I still got 3.675% for the remainder of my 15 year mortgage (12 years left).
I think NFCU was able to do this because they do not sell their mortgages and they service all of their loans in house.