First to go out on a limb with a preliminary presumption. Most couples inclined to swinging type activities , will already have highly developed sex life with the two of themselves. A new playmate would be comparing against the cumulative experiences with the spouse (well that's how I'm seeing the question, and and feeling about the matter).
New to you, and different is always interesting, and usually good for its own sake. A new playmate might do something you haven't tried before, do something particularly well, or physically the respective parts (not just genitals) might fit together in a different way. But for a new or occasional partner to even be compared on same page with spouse, either the new person is a world class lover, or the home situation is in need of serious remedial attention.