Hiya Sealovers.
I think you've been given some great advice here that’s probably spot on. I'd like to offer a slightly different take though, it's probably not the right advice for you but since everyone handles things differently it may be a helpful perspective.
Lighten up. It's sex and it's fun.
It sounds like you are pretty "in your head" right now, and that’s a great place to find fear and insecurity to get in the way of an otherwise really enjoyable experience. If your wife and you both think this experience would be fun, why not give it a try and see? Since the lines of communication between seem to be pretty wide open, odds are it wont do any irreparable harm to give this a shot.
If you try it, and find the jealous feelings inhibit your good time, then you know it's not for you yet.
If you try it and find the sexiness of the whole thing completely overwhelms any other feelings, then you know you are in the right place.
As long as your wife and you are communicating freely and without inhibition, then I suspect you are pretty well insulated from any lasting harm. You don't have to be perfect or be in a perfect relationship to enjoy swinging, you just need open communication, a sense of perspective (it's just sex) and a willingness to go and have raunchy fun sex with other people.
as I said, this may not be the right advice for you but it sounds like you have plenty of reasons to hesitate already. I just wanted to give a word or two from a different view.
Whatever you choose just be sure it's what you and your partner really want, and be sure you both fully understand the others reasonings.