Instead of plane fare and hotel expenses, rent a small cottage or cabin in a area that appeals to both of you- perhaps within walking distance of the beach/lake or a quaint village/town with ammenities and a romantic cafe.
Honeymoon is an expensive affair. But if you really want to have cheap honeymoon go to some place which is not far away from your home, so that you can save on travel expenditure. Also choose a budget hotel, and the places to visit may be like beaches, parks, waterfalls etc. where no admission fee is charged.