The persistence of a particular desire can take over your life, to the point where judgment is suspended and the potential consequences of your actions are a dim spot in your brain. I also am a look at the deeds and let the words wash sort of person and I'm seeing desperation on his part and non-acceptance on yours. Unlike Lionheart72, I don't think you're the victim here, or at least not the only one. What if you had said, "Meh, the idea of another man touching me is squicky, but I see that you really, really want to do this. Would it work for you if I just watched?" Your outcome might have been very different and you wouldn't each have violated trust.
I can't get quote to work for me this morning, but in order, here are my answers to your question:
Forget swinging. It's a crap answer and has huge potential to cause more problems than you already have.
Yes, I have. At the start of our relationship, my lovely man told me, with great trepidation, that he wanted to try sex with another man and have me there. But he didn't think he could stand having anyone else touch me. There are any number of men on CL and elsewhere who want just that, so not a problem.
You don't actually enforce rules and boundaries. Enforcement is a parent to child thing.
If you don't want secrets, don't force desires into hiding. My answer might be different if he had a habit of secrecy, but I'm hearing that it's just this issue.
You rebuild trust by looking at the complete picture, acknowledging your role, including how your fears are in your way and in his, being compassionate in your anger and talking, talking, talking.
Swing clubs are not generally friendly to M/M contact. There are exceptions and there may be a club in your area for which that's not true, but you'd be better off getting a profile on SLS and finding potentials there. I don't think it's a crazy idea at all, btw, although it is a little unconventional. However, first you need to do a lot of work as a couple, including really talking about your fears of his leaving you to embrace gayness.