Thanks all for the advice. To answer a couple of questions. No there wasn't the option for penetration, we don't full swap, so no penetration of the other wife and my wife wasn't in position to be engaged. The other couple is actually very experienced. PeterJ, I think you hit the nail most completely on the head. I had some felling or thought that the other couple and even my wife should know that I wasn't involved in some way. I realize that isn't the way to think because people can't read minds.
In the end I wasn't enormously upset, you have to understand that the way I deal with stuff is that I tend to think on it extensively for a couple of days to determine what I feel. Don't know why, that's just how I process. My wife and I spoke more about it yesterday and came to realize that it was really just a breakdown in communication. I assumed my wife should know when I've finished, but apparently, I'm not as vocal when I finish as I thought. And further, I apparently have different sounds between oral and intercourse. So, there was no way she would have known unless I flat out told her. I didn't interrupt things or try to squeeze up in there and maybe I should have tried to push in more and make my own space. That said, we determined that if we are in a scenario where we sense the other off to the side or out of the game a little, we will make eye contact, and / or the one of use that has gotten kind of left out a little will give a little light double bootie tap as a silent sign to switch back to each other. Lastly, I do think maybe there was a twinge of jealousy in there....lets be honest this is the first time in 25 years my wife has gotten another man off...and certainly the first time I've ever seen her do it. It wasn't unexpected, but hadn't happened yet and I had to process that emotion too. Let's be honest though it was hot, she gets a little look of control on her face while she's doing her thing and it's definitely awesome to watch.
At any rate, we are good. I needed that extra talk and I should have been more completely honest with here initially. The problem was that I thought I was okay, it just took a couple of days. Thanks again all!