I have often wondered that myself. ^^^
Then I read an interesting study.
They asked heterosexal men how comfortable and anxious they are around gay men. Based on these scores, they then divided these men into two groups: men that are homophobic, and men who are not. These men were then shown three, four-minute videos. One video depicted straight sex, one depicted lesbian sex and one depicted gay male sex. While this was happening, a device was attached to each participant's penis. This device has been found to be triggered by sexual arousal, but not other types of arousal (such as nervousness, or fear - arousal often has a very different meaning in psychology than in popular usage).
When viewing lesbian sex and straight sex, both the homophobic and the non-homophobic men showed increased penis circumference. For gay male sex, however, only the homophobic men showed heightened penis arousal.
Heterosexual men with the most anti-gay attitudes, when asked, reported not being sexually aroused by gay male sex videos. But, their penises reported otherwise.
Heterosexual men who reported having the most anti-gay male attitudes were more aroused by gay male pornography than heterosexual men who had pro-gay male attitudes.