First off...welcome to the forum though I'm sorry you had to come under such awful circumstances.
Second, and I came to this conclusion 1/4 of the way through your post...drop him. You two aren't on the same page. It sounds like you two haven't even communicated about swinging and just going out and doing it...ask any successful swinging couple here and they will tell you that communication is very important. Trust and respect is another. Couples need to talk, need to trust, need to respect, and look out for each other's wants, desires, fears, etc. From your post, I see none of that.
And I know you're in love with him but if you keep going with him things will turn out terribly for you. He gets mad that you don't agree with what he wants to do. He calls you boring when you don't do what he wants to do. He is being forceful in getting what he wants. He isn't being truthful to you about his escapades. If you heard your girlfriend talk about her boyfriend treating her this way, what would you say? You'd probably say that he's not a winner. Relationships require a give and take between both partners and it sounds like he's taking, taking, taking...and not giving. He's not respecting your boundaries. He's not listening to your fears and sense of discomfort. He's being very selfish. And that isn't helpful to any relationship unless you're in a relationship of one.
Please forgive me, after the first 6 paragraphs, I only skimmed the rest of your post because it was actually painful to read how you have been treated. I think deep down, you know he's not the "catch" you want to believe he is. Walk away. If you really want to explore and learn about the swinging community, please stay around the forums and poke around. And then, if you feel that maybe you are interested in swinging, maybe you can find a partner that is interested but also willing to treat you as a thinking and feeling human being during the entire relationship and swinging experience. Stick around long enough and you'll see that a lot of couples move at the pace of the slowest person--the one that is more uncomfortable. Couples where one half forges ahead and leaves the other half to catch up end up very badly if they don't stop and rectify the problem.
I wish you luck in which ever way you choose to go from here.