First I feel compelled to say that I have tried the "Hold off" for a day or two method and it didn't work at all for me.
That said, I have made a few observations over the years regarding this problem.
For me, the number one reason why this happens is that the woman I am with is just not doing it for me. This does not mean I do not find her attractive or sexy. What it means is that her idea of what sex is and mine are not in sync.
A good example of this is our most recent swinging experience. We met a couple through SLS and agreed to meet at one of the local clubs. They were new (had only had one previous play experience), but we didn't let that deter us, we just spent a little extra time getting to know them to make sure they both seemed into this and on the same page, as we usually do with newbies. We thought we had thoroughly covered each others rules before going to the play room, but we were wrong. Once we got to the play room and got started, the woman said, "oh, by the way, no kissing is allowed". Now normally we would have ended it right there, but it was a slow night at the club and they were really nice folks, so we pressed on. Next surprise, she says to me, "oh, don't play with or suck my breasts, I just hate that. And while I am on the subject, I really don't care much for receiving oral either, it just doesn't do anything for me". OK, at this point, I was beginning to see that this was going to be a "less than optimum" experience. Sure enough, while everything came to proper attention while she was giving me oral, by the time the condoms were on, the condom was quickly becoming unnecessary, as I was deflating rapidly due to a lack of stimulating ways to occupy myself. So, at this point we called it a night.
The funny thing about this encounter to me is that it kills the notion that a lot of folks have, that all guys are turned on by the same things. Because, other than the kissing thing, which this couple claims they like to do with each other. After talking to them, it turns out that he doesn't care for playing with tits or giving oral to a women. So, while they are a perfect match for each other, they weren't even a close match for us. The funny part is, because he isn't into these things that she doesn't care for, it never occurred to them to mention them until play was underway. In other words, because these things are not important parts of sex for them, it never occurred to them that these things would be essential for someone else to be sexually stimulated.
The second most common reason for me to have a problem is being tired. I can be all raring to go when we head out to the club. But then by midnight or later when it is time to perform I am more ready for a nap than I am for sex. So, to mitigate this problem I have often let it be pretty well known that if one wants my "A" game, we better get started early. In fact it is an often heard comment around the club that GT turns into a pumpkin at midnight.
A distant third is to many distractions, over time though I have gotten to the point that I can usually shut the distractions out. This seems to be the key, at least it was for me, experience seems to be the only way to get over this one.
Lastly is the problem of defeating ones self. What happens is that you worry that you may have trouble "keeping it up", and because you are worrying so much about it, it becomes a self fulfilling prophecy. It is easy to say, "don't do that" but my experience is that this too is only overcome with experience.