Feelings are very hard to control. Try controlling your reaction to the feeling rather than the feeling itself. Evidently it is causing you to do things that you don't want to do. Without knowing your age and other things about you any direction given is going to be limited based on what is known,

Find a negative aspect about the guy that irritates you and blow it out of proportion so that it consumes his attraction.

Always works for me. For instance, if a guy's good looking, I'm hot for him usually. But as soon as he opens his mouth and is disrespectful/rude/disgusting/repulsive/scummy, he's not interesting at all anymore.