Lots of good thoughts in this thread. I'm a single female, so take this for what you will.
To me the betrayal isn't in the physical/emotional, it's the deceit. The purposeful and willful lying to a spouse, either with words or by omission. I would be more hurt by an emotional connection, but if it was physical and the person decided "I'm going to fuck Susie Swinger three times a week and never tell Des", that would be a huge breech of trust and a lie by omission, unless our relationship was open and we didn't even have to talk about it.
It's different if it's boundaries crossed and admitted to and talked about. Blowing past boundaries and never looking back, admitting to it or talking about it with the deceived party is where the cheating comes into play.
Cheating is about deception and hiding. It is disrespectful and unloving at its very core.