I'm just a hick Okie, Van, and I can't say I've had any experience with Polyamory, but it seems to me that you have two choices ... the four of you can agree to swap partners and experience that which you've long wondered about, or you can continue living as friends as before, pushing this phase under the table to never mention again. The positive part seems to be that nobody is willing to sacrifice the marriages or the friendship.
But the decision needs to be made by all four... together ... openly ... by talking out fears and feelings. Without that, yours will be less than the friendship y'all once had. ...and by the way, there's no reason that the four can't research polyamory together just like a couple might investigate swinging. The decision doesn't have to be made yesterday.
Good Luck!
Alura