Not at all ! There are quite a few of them now and many are Pros. Also there is the IFBA . Go for it !
You should look into some martial arts. So many different ones. Karate, Japanese Jiu Jitsu, Brazilian Jiu Jitsu and many more. You learn to defend yourself, not be the aggressor. I've studied Japanese Jiu Jitsu over 40 years. Takes discipline and patience. Call around.
It depends on what ways you want to fight like Boxing or Taekwondo. I took TaeKwondo for about a year and now I want to take some boxing classes (mostly for self defense).