I would never have been a swinger.
I became a swinger.
I no longer am a swinger.
I have the perspective of all three stages and can shed some light on this and hopefully save you from a lot of heart ache.
As someone who was brought up religious, I found it unconscionable to consider swinging or cheating. I would have a dream I cheated and feel so horrible for days for having the random dream. So how did I go from that to being a swinger? Easy... my needs were no where near met especially in the bedroom. My wife was passive about sex, dull and there were many gaps of over 3 days. That is my and many men's magic sex gap number. Beyond 3 days is creating frustration and bad feelings. It really depends on the mans sex drive. But a man's satisfaction will be measured by overall time spent satisfied vs frustrated.
The idea that quality is more important that quantity is bunk for men. That's like saying you can pee once a week but its going to be REALLY good. It's a biological urge that doesn't do well when held back beyond that persons physiological needs.
There are 2 ways you MUST approach sex. Quantity Evaluation and Style Evaluation. If you feel that things have changed, it's because as a man ages, he becomes more specific in his desires be it sex, fishing, sports or work. He knows what he likes and wants.
Firstly, quantity is easy. If he has sex, he will have a satisfied period afterward. This can shift slightly with his bio clock(60-90 day cycles for me). What you have to do is make sure you have sex before his frustration kicks in. I suggest every other day. Give a hand job, bj, something... It's deadly important that his desire is fulfilled BY YOU. This keeps him from ever wanting porn, fantasy thoughts and cheating INSTEAD of you. The more you fight being the meeter of this need the more he fixes the problem with a solution that meets his need. This is a mechanical thing men do. Problem->solution.
Style is tricky, but in your case I am pretty sure I know whats going on. He loves wild sex. Lingerie ain't gonna cut it. He needs you to be slutty with him. Not really, but as a persona. As examples, beg to swallow, depict a level of desire you can't control for him or just be more noisy and athletic during sex. This is what he craves. Swinging just does that in spades. He also needs taboo. Go parking and have sex. Do it on your deck at night... BE the taboo but do so safely and with boundries. Never say no. Give alternatives.