Self-Defense for Anyone
Royce Gracie won UFC1, a no rules fight tournament where he was the smallest and least athletic competitor, and won easily.
Success as a martial artist is not entirely dependent on your size, athleticism, strength or speed. These attributes can help but are no substitute for proper technique and anticipation.
Join the Martial Arts Community
Martial artists make up a very racially, culturally and socioeconomically diverse group. You will meet some good people while training.