We had some similar things happen like that with us. I played with my husband for a long time in RP with our inventory of playthings thinking, I wonder what it'd be like if we tried it with real people? Over time introducing more and more ideas to each others we had a discussion about it deciding to venture out and see what the lifestyle is like. Scared the jeeperz out of us at first but also equally turned us on. It took a while longer but we eventually became acclimated to being around others having sex, building friendships, trust, and finally started having sex in the same room as others. Then I had my first friend, then another, and another. It took me three times and I was all in! My fourth and fifth I became wilder. Scared my husband. He settled down but only had sex with me in between me having sex with others. As time went by he and the other men in pairs would love on me together. I thought I'd hit the jackpot. Then got to know three bicouples, a whole other level of scared the begeebees out of us and yet somehow attracted us to them. They were so real, so care free, the sex flowed, no hindrances, Wow! The combination of their personalities, lifestyle, and sex won us over. When they finally went all out same gender I thought my husband was going to get up and leave he was sweating so bad and his eyes were so wide I thought they'd pop. I was the first to dive in (yes he was okay with it - all same room). A husband here, another there, then my husband and another husband. Again with winning the jackpot! I was so into the next few months it was most of what I talked about. Husband was overwhelmed but never said stop so we would meet once a week as a group for a few hours. The wives asked when they were going to get to be with him. I let them know he was on his own time table.
Skipping way ahead to the next Freak Out - Mine. The group grew with more wonderful friendships, all bicouples. More went on and one day we were able to break through a fantasy of mine, my husband even enjoyed it. Time went on and something clicked in him, this time he went gungho. Scared me. So even in the incredible friendship, loveship, and comradery, we still have our freak out moments. We always dialed it back, moved forward with restraint, and all works itself out. We never do anything without the other present even if one of us is not participating we stay in the same room. We stop if one of us needs to leave. We never ever are alone separated with anyone, everything is all same room even shower sex. We found our balance, what works for us. When I get overboard, he protects me from myself - Yes I can wear myself raw. I do the same for him, I know his limits when I need to say, "You know how much you've had in the last hour?"
Perhaps you two need to dial it back a bit, then proceed cautiously with whatever your guidelines are. Find your balance.