Dear messedup; I'm new here so I will be as kind as I can be, because mostly that's what I would like.
First off think about what you thought swinging was going to be like when you first entertained the notion. Now realize that it's not just fantasy, but reality. It's not just you, and your man in you safe place calling that shots just the way you want them, then cleaning up, and going to bed. (we learned that one that hard way).
Second thing you started this! Your husband is doing all he can to keep you happy while not getting lost, or making bad decisions.
Third thing is you did not start off telling us about you being the queen of the BJ as far as you knew. So I think it is safe to say there is other things being held back from this forum, as well as your therapist. I know you say your being up front because of the money. But, what ever.
Forth, you have everything you need to get through this if you just stop trying to be so damn rational. Ask yourself the question you don't what to know the answers to.
Fifth, if I was to guess I would say without a doubt you are a control freak. On the night in question you not only didn't have control over your husband, but even worst you didn't have control over yourself, as you have stated. Now your just plain old pissed off, and you are making your self fell better by saying that you don't know who to forgive, and move on. Control is an illusion!!! But you are working so hard to prove other wise you risk all you have going.
No one here has said it yet, but I will, you may have turned a corner that night for one reason or another, but your husband is going to reach a corner of his own soon. He didn't force any of this on you! People can only take so much. Please for the sake of our children, ask yourself the hard questions. Surrender to a lack of logic. You entered into a place of no life reference. Your trying to get answers to questions that aren't asked. You can't turn to you friends or family. For the most part your alone on this in your day to day life. If your driven by logic, then write your reasons for getting into this, all your expectations down. The things you did and what you wanted, and work your hard questions from there. Keeping in mind that expectation leads to disappointment.
In ending your in a bad place rite now, but when you work your way through it you will be the better for it. So often we treat the symptom, and not the patient. Chances are your are going to have to deal with other issues as well as what is here now. When under pressure a good rule of survival is don't think about everything at once. One thing at a time is best.
Wish you, and your husband well.2C