For me, their is no such thing as too hot, but like some of the others here have mentioned, I have met a few women that put off an attitude that was a turn off.
What I have observed at the clubs is that some women are more approachable than others, and it really doesn't seem to matter how hot they are. Not knowing you personally I can't say what the problem is, but it very well could be that this single gal you took with you just puts out a more approachable vibe than your wife does. That doesn't mean your wife is not a nice person, it is just one of those things I have noticed, that certain people seem to have a gift of giving off a vibe that encourages people to approach them and others don't.
As far as dressing down goes, we have some experience with that. We normally dress up to go to the club. At one point we noticed that most other club attendees were much more casual about their look than we were. We were in a dry slump at the time and thought, what the heck, lets go casual for a while and see how it goes. After giving that a try for about two months and having no better or maybe even worse results, we went back to dressing up nice and that has seemed to work better for us. I think at least part of the reason for that is that a large part of being approachable is the attitude you project based on your self image. For us at least, we feel a lot more sexy or attractive when we dress up, and I think that in turn causes us to project a more sexy attitude, which I think has a lot to do with our better luck at hooking up with folks when we dress up compared to when we go more casual.