I have to echo those that think its pretty horrible that this guy is already worrying about his own libido when his wife was diagnosed with cancer just six months ago. OK sure, he "stuck by her for a few months" and maybe now she is in remission, but my goodness... The experience I have had with loved ones who have had cancer hasnt been that the second they seem like they'll survive Im back focusing entirely on myself! This guy sounds like a real asshole, to be honest.
That said, what do you think your wife meant by "he's my only toy"? Thats a very odd comment considering your situation (multiple playmates, long time in lifestyle, great sex together)
I hate to throw a dark cloud over you, but I'd really be on the lookout if I were in your shoes. I also think that a total pause from the lifestyle while you examine exactly what the heck went wrong here might be in order. I would give her some space for now and not press, but I would also just quietly stop all activity in the meantime.
I know that you feel comfortable that the only problem here is that the other guy is hiding something from his wife, but from the outside based on what you've revealed, it looks like there is more going on than just that.