Early in adult life, there is exploration and experimentation. Young people are expected to "date" widely. 100 years ago, virginity might have expected. Contemporary cultural norms reflect the fact that a couple should probably "try out" all facets of married life, including sex--and especially sex with each other--prior to marriage.
There is a wonderful romantic myth that we all eventually find "the one" true life partner. The wedding vows typically involve "love, honor, and 'forsake all others'". The problem, of course, is that people grow, they grow differently, and sometimes grow in different directions. Monogamy, which is used as a symbol or as a fence, is threatened. Boredom ensues, and the married couple starts to to yearn for some variety. What's interesting is that so often the couple is still very much in love. The marriage may also make sense for economic or other reasons. The couple wants to sustain and even further mature the relationship. But sex--or more precisely lack of sex--is in the way.
For women especially who have been told that after marriage, only marital sex is permissible, start having difficulty dealing with their sexual fantasies. They have been told it is "wrong" to have them, much less act on them. Men seem to have less embarrassment about the fantasies, which itself is problematic because it can make the woman feel somehow inadequate. Still, the idea of sex outside of marriage has been intrinsically thought to be so sinful that it gets not one but two mentions in The Ten Commandments: thou shalt not covet they neighbor's wife, and thou shalt not commit adultery. (Killing only gets one mention, which gives some sense of priority. ) So it is understandable that consideration of (much less a decision to) violate 2/10 commandments causes all sorts of stress.
Of course, swinging is neither about committing adultery nor is it about coveting someone else's spouse. It is adult play. But it is hard for many--men and women--to wrap their heads around the idea that it is just adult play. And if something "goes wrong" , there is fear that there will be irremediable and horrible consequences. At the same the are looking forward to some play, some fun, some relief from the ordinary stresses of the ordinary world. "You make the arrangements, I will just show up" is another way of saying "If it doesn't work, you own the responsibility, and I will not burn in the fires of hell".
We might suggest finding two or three lifestyle couples who seem interesting enough that you might enjoy--and especially that your wife might enjoy--sharing drinks or dinner without any expectations beyond that initial social get together. We spend so much of our lives these days isolated that the idea of actually getting dressed a bit to go to a restaurant and enjoy someone else preparing a meal while having interesting conversation is going to be attractive. The nice thing about the lifestyle is that "no thank you" is respected and never taken as an affront. On the other hand, the mutual seduction is fun and interesting.
What makes the lifestyle good for many who are in it is knowing that their spouse is quite attractive to others--but going home with you.