My sympathies @waningmoon. Being betrayed sucks, I hope you are seeking solace in friends and family that can help you heal.
About your situation, there are two types of non-monogamous relationships, ethical and non-ethical. You've just discovered (to your dismay) that you've actually been part of a non-ethical non-monogamous relationship. Not by your choice obviously, but definitely by his.
I would guess that this is a genie that you cannot put back in the bottle. The foundational expectation of your relationship - that you would be sexually and emotionally monogamous - was not in fact a shared by both of you.
So, what do you do with that?
Well first I encourage you to heal and get over your anger and grief, give yourself the time to do that, and you don't allow yourself to get pushed to 'put it behind you' etc, until you've truly processed this pain and betrayal fully.
Then you need to decide what you want to do about your relationship with this man.
He has revealed to you that not only is his assumptions about your relationship different than yours, but he's also willing to act on his assumptions regardless of whether you are on board or not. You need to decide if you are willing to be in an 'open-ish' relationship with him. If so, he has to decide if he's willing to abide by some ethical guidelines if that is the case. Open relationships can range from fully and equitably open, to 'don't ask don't' tell. The choice is yours (and his).
However, that may be a bridge too far, and if committed monogamy is the only kind of relationship you can see yourself part of, then my gut says this relationship is over. He can promise to be better, but really I think that would just mean better at subjugating/hiding/or disguising his true desires and behaviour.
D