OK, my advice is set the swinging/sex idea aside for a moment. You say you have anxiety issues. Think about how you usually deal with those?
Just think about it in terms of going out to a new place, with new people. Think about your standard coping mechanisms. Turn to them and apply them here.
Now, once you've thought of those, we can open up the idea of sex. The idea of sex is, I'm sure, adding to the anxiety. So I guess my advice here is, give yourself permission to NOT have sex. You don't have to. You've already gone to the club a couple of times, not had sex and had a good time. Do that again. If, while you are there, you and your partners all feel that the mood is right for sex, go for it. If any of you... especially yourself... don't feel it's right, don't do it. If you started, and it feels wrong, stop. Just say "I'm not feeling this" or "I'm sorry, I don't want to continue right now" and step away. That is OK. It is always OK.
The key here is don't put pressure on yourself to go and have sex. Go and have fun. If that fun involves crazy acts of naked passion, great. If it involves sitting around chatting with your partner, great. Whatever works for you on the night.