This is a longer answer than what you might want, but the rationale matters.
What happens inside a marriage and how two people grow within the marital bond is their choice: there are as many choices as there are marriages (no two are alike).
Remarks such as the one your "vanilla guy friend" made come from one of three places: envy, curiosity, or disapproval. You are not obligated to respond. It is your marriage, not theirs.
But if we were to respond, we would think to ourselves that "vanilla guy friend" cannot deal with the notion of vulnerability-his, or that of his marriage. He sees monogamy as a fence to 'contain' what is 'his'. Either he wishes that he could be comfortable with your acceptance of vulnerability, he is curious about how it feels to be vulnerable, or he disapproves of 'a guy' (or for that matter, anyone) exposing themselves or their marriage to perceived risk.
Place that in context of what really goes on. He has a fantasy life, his wife has a fantasy life, likely unshared--and that makes them both vulnerable. Our response -- if we chose to give one--would be along the following lines. "One of our core values is honesty with each other. Part of that has always been sharing our fantasies. Along with those fantasies, we set intentions, and boundaries. So doing, we create a choice to act on those fantasies--or not. It keeps us honest, and our marriage is stronger for it."
Where he's having trouble, most likely, is seeing the concept of monogamy through a different lens. Is it an emblem of the marriage? A fence to contain the marriage? A fence to protect the marriage? An expression of "property" rights?
"Long-long" couples---long married and long in the LS---view the marital bond quite differently than most others, and yet that view of fierce loyalty, of 'team together', and of genuine "'til death do us part" is remarkable in its authenticity.