Trace, I think you have to clear your mind of all the 'what if's' and just meet to hear what she has to say and cross that bridge when you come to it. Obviously you're on your toes right now ready to walk on egg shells, but I think your response to her filling you in on the rumor will have to be in how you perceive her tone; judgemental, intrigue, curiosity etc.
My first train of thought since you aren't close with her, and therefore she wouldn't care about you enough to discuss any rumor she heard, is that she may have heard a rumor that you are swingers, is curious and interested in exploring it further, but you want to be very astute and tight lipped on it until you read her body language on it and why she's talking to you about it at all.
Lee, I have to disagree about being honest and open only in the context that business could be hurt causing more of a hardship on the family if clients pull out and sales are lost. Trace, if you perceive, at all, that in your business circles this could hurt you and your livelihood, then I would simply say that you've picked up on the rumors too, but they're not true; it must have started because you have some friends that are swingers.