Of course not! I don't think anyone is suggesting you were.
Your first question in this thread was "Am I wrong that this bothered me?" That's a reasonable question to ask, and I think we've all helped give different perspectives on the subject.
There's another perspective I'd like to add on. My wife and I discussed this thread last night. We frequently discuss interesting threads from this forum. One of the best reasons is it helps us increase our swinger IQ so to speak. We learn by discussing other unusual situations and can then have a better understanding of how we feel about it.
So anyways; what we both kind of concluded was that there was a sort of "she said-he heard" difference in gender perspective situation that occurred. Neither of us would ever want to convey a gender stereotype. But, your husband may have heard things in a pro-forma check box kind of way. You've expressed in abstract it's ok for him to play with others, and said so before this evening. Check. You met another couple, of whom he was attracted to the wife. Check. Both couples began playing, so there's obviously some comfort level among the four of you. Check. He is invited by the other couple to play with the wife of the other couple. Check. He asks you if it's ok to go ahead and do so, and gets agreement. Check. He goes and plays with the other couple's wife.
All hell breaks loose and the guy is left bewildered wondering "what the hell did I do wrong?"
Meanwhile, from your perspective maybe; it's yes, you've given permission in abstract. Yes, this couple is nice. Wow, I'm really enjoying what my husband is doing to me, and it's cool other people are watching me...damn his tongue feels good! What? The other couple wants my husband? Oh that's ok, he won't rush over there just now, not just now, please not just now. WHAT? He's going to her NOW? Leaving me hanging? Oh I don't want to look like the cold bitch from hell, but why wouldn't he finish with me first? Damn it, now she's getting what I wanted and I'm left high and dry all alone on this bed. Why didn't he want to stay here with me? Didn't he understand I needed him right then? Couldn't he have finished with me first? Oh I'm steamed and hurt!
None of which internal self-monologue was heard by your unsuspecting husband.
One of the worst things one person can do to another is fail to live up to the other's expectations of them. The hard part is the expectations are rarely vocalized.
The outcome of all of this?
Neither of you is wrong. Neither of you is right. You have every right to feel as you do. Your husband has every right to feel as he does. The important thing is that it seems you're both willing and able to talk about it, and make it better for next time.