Perhaps the host and other couples suspected that the OP's husband was being aggressive in keeping the single girl playing with him? Not saying that he was or wasn't, but they may have perceived it that way, especially since a number of experienced people in this thread have mentioned that they have seen that many times before.
Personally, if I was wanting to play with someone (single or couple), but never got the 'opportunity' because another person/couple were with them I suspect I'd be annoyed at the end of the night....but it would be at myself for not taking the initiative and either asking to join in, expressing my interest to play or whatever. Old insecurities may creep back in and have the ego get hurt for the cute single girl being into someone else other than me. All of that would be my own issues.
Now, if the guy actually WAS being aggressive in keeping the single girl there with him then maybe they all have a reason to be upset with him. Even still, the single girl needs to make her desires known and seek the host couple if she needs help.
All of that is from my limited experience though so take it with a grain of salt