CountryGal - I am so sorry your foray into swinging has resulted in such heartache and pain in your marriage. It is often said that swinging magnifies a relationship - if you're happy, you'll be more happy...but if things weren't good to being with, then that is magnified as well.
At the risk of taking a position that others have not voiced, I wish to clarify that it appears your husband was a willing participant in this event. At any time during the planning, discussing, chatting, or the actual sharing did he flatly say "no"? While I read your comments about agreeing to take it slow, but that you raced ahead and wanted to push the timeline along - what I didn't read was any report of flat refusal on his behalf nor any strong words about the subject at all. I feel that he easily (and it was his responsibility to)could have said "no, enough of this...we will not swing" or something of the like at any point during the past few months right up to and including arriving at the hotel - even during play.
I guess what I am saying is that you are BOTH responsible for this problem. It seems to me he is projecting the blame on you and very much playing the victim in all of this. If he wants to save your relationship, he will need to resolve his issues and talk to you soon.
I think you have chased him enough. Either he wants to fix this with you or he doesn't. I liked the suggestion Julie gave about writing a letter and including a scheduled appointment time for a counselor. With or without him, you need to talk to someone about this event in your life...
I am not a man, so I can only imagine what feelings he is experiencing that are causing him such angst. However, I am a woman and can identify with your side of the situation. You are not responsible for this situation alone, nor are you responsible for the way he is processing it.
I think you have done all you can while still being true to yourself. Not to be so cavalier about the institution of marriage, but at some point you have to ask yourself, is this worth saving? What will things be like in a year? Will you be walking on eggshells just to save his broken heart?
Again, I am sorry - and I apologize if I haven't read/interpreted the chain of events clearly. Perhaps my point of view isn't what you need right now, if that's the case, feel free to disregard it altogether. More than anything, I hope the two of you can work this out and be stronger for it.