This is true. Many years ago, my wife and I accepted an invitation to a meeting at a remote mountain cabin from a man and a woman who were married -- just not to each other. We accepted the invitation and subsequently had a couple more nice play dates. About a year after the first meeting, a call came from the woman saying he had been found out. We wished her the best of luck in future endeavors and knew we would never hear from either again.
So here is the thing. We made it clear that if either or both got caught that they should not expect either of us to get onto a witness stand and tell lies. They understood and seemed to have already considered their risks. We considered ours. If we encountered a situation like this again, we would consider it on its merits.
We have encountered and heard of other married-but-not-to-each other pairs. Although not having played with these others, we don't walk circles around them or treat them like outcasts in social situations. Unless you know the whole story, how can you judge,