Would I have to go through any math, physics, or chemistry courses to get a degree in animal behaviorism? I love psychology but I find it difficult to enjoy the three other courses I just mentioned.
Almost certainly. If you want to go into a field in the sciences but don't want to take science or math, you're in for a rude awakening.
Most of animal behavior is biology anyway, not psychology.