Interestingly enough, among the GLBT community, bisexuality more applies to people who have emotional as well as sexual relationships with people of more than one gender. In the swinging community, you find very few people willing to admit that they have emotional feelings for members of the same sex.
So, to hear someone say "I'm not gay because I could never be emotionally attached to a ," is rather insulting to those who are bisexual and consider themselves bisexual because they enjoy both sex and some level of emotional attachment.
Kinsey rated it strictly on sexual activity. Masters and Johnson got a little more fluid (heh) with it. More recent sexuality researchers got even more complicated.
People who identify as bisexual because they can enjoy both sex and romance with any gender are often frowned upon by both gays and straights and spend a lot of time getting told what they are (well, you're really a closet fag: you're really just gay but repressing: you're really...)
All this goes to prove is that sexuality is a very complicated thing which defies easy categorization. Which is good and bad. Bad because the folks who like to keep things simple and have definite categories people have to fit in get frustrated because not everyone fits in the box and it annoys the folks who don't exactly fit any of those categories. Good because lots of people feel that narrow categorization is a Really Bad Thing and spending some time to get to know who a person is is rewarding.
Me, it's easy. I'm bisexual because a person's gender is irrelevant to me. I become physically and emotionally attracted to who someone is and how they present themselves. In my life that has happened far more with women than men, but that's because my attraction is based on a mix of factors, including whether that attraction is mutual or not, which is more likely to happen with women than men.
So, to the OP, if you like sex with men, enjoy! If you only like sex with a specific man, enjoy! If you feel emotional attraction, enjoy! If you don't feel any emotional attraction, enjoy the sex! If after you've satisfied your fantasies and don't feel like you need to do more with other men, don't worry!