It's not a dumb question at all. The short answer is yes, there is such a thing as two couples being in a relationship.
As to how it works, that really depends on the relationship. We have had a poster here on the boards who is in a polyamorous relationship with another couple and, I believe, a single women. They live together and even have children together across couples. On the other end of the spectrum, we have swinger couples here who become friends with their play partners and only play (have sex with) people with whom they have some kind of friendship - whether casual or close. What is important to remember is that relationships that involve more than people can be very complex. If you have four people, you have at least six different relationships - even if they aren't all sexual ones - to balance and maintain. Depending one the depth of those relationships, it can take a lot of time, a lot of work and a lot of emotional energy. For those who can do it, it can be very rewarding. For those who can't, it can cause problems.
Are you over thinking this? You know, whenever I ask myself that question I already know the answer is yes. I think you do too.
Are you setting yourself up for a terrible situation? Honestly, that depends a lot on your situation. The core of both swinging and polyamory is this: open, honest communication. Can you talk to your husband about anything and everything? Every fantasy, every fear, every foolish hope and crazy dream, every emotion that you feel but tell yourself you shouldn't, everything you are afraid to say, every dumb question? Can you, if you needed to, share all of it with him and will he listen? Can he share all of that with you and will you listen? If you can, you're off to a great start (and way ahead of where I was at 25.)