It can since sexual arousal is an altered state and you do not filter stuff as usual. That and classocal conditioning (associating pleasure with a trigger behavior) can make anyone go that way. It is true though that some are affected more than others.