The conventional wisdom is that men are more visually stimulated with sex than women are.
I think this is, in general, a true statement in most circumstances. Women look for status, power, wealth, or attributes which can lead to such things such as intelligence, confidence, charisma. This is basic and genetically driven. Before someone tells me they only like weak poor shy men and married one just like that, these are just general trends. Like all trends there will be people on the ends of the spectrum. Its not that mens looks don't matter, but they are secondary.
Men look for youth and beauty first and foremost. This is then followed by other traits which tend to rate of a lower importance. I think this does lead to a rather amusing karma when very wealthy and powerful men marry the very attractive, but maybe not so bright woman and then wonder why their kids are such losers, forgetting their kids are 50% her. Again this is a trend, maybe you only go for ugly old women with multiple PhD's, but as a rule this is rather obvious.
Somewhere along those lines we tend to find people 'our level', end up getting married and hopefully are happy.
This is 'normal' life, normal dating, normal behavior for long term relationships.
Swinging is different, well kinda.
For the men, its pretty much the same. Men want youth, men want beauty, men want HOT. You can get into 'types' and the like, but it doesn't take a lot of effort to see what women would be the most popular at a given event. Yes personality does matter, but really good looks can make up for a lot of bad personality generally easier than the other way around. Yes we all have personal examples again where the girl you didn't find attractive had such a great personality etc but trends.
For the women, I think it changes a bit. Women are not looking for a long term relationship in swinging, women are looking for a fuck toy. Liking the guy is fine, having a good personality is fine, but they don't have time to find out his inner strengths, and it really doesn't matter to them much either. Looks are more important than in 'real life'.
I keep bringing up that this isn't how everyone swings/rate partners, mostly because in the past people will take their exception to the rule as the rule. Just because maybe you as a male only care about a womans personality or you as a woman only care about looks, it doesn't change the trends which are obvious in society.
On a personal note, the focus on looks in swinging by the women is why I've been hitting the weight bench and dropping the love handles. If I want to have sex with women I find sexually attractive, I need to be physically attractive to them as well.