You have to face your fear. By the way, that applies to pretty much everything in life. Because if you live exclusively in your comfort zone and can always be controlled by your fear, there is little room for improvement.
That does not mean that you should start a bar fight tomorrow or any other nonsense.
Rather, I'm concerned that you put your fears bit by bit. That's the only way you can learn to control your fears and keep improving yourself in every area of your life.
Start with small, rather simple things first and increase with time. As a result, you get to know yourself much better and become the true master of yourself.
Especially with the fears of a physical confrontation on the street, you can learn how to deal with how to train as realistically as possible.
During exercise, we usually do not have situations where we exercise under high stress levels. We know our training partners and know that we drive home safely after training.
The training should be fun, of course, but for me a certain seriousness and even more important, a certain aggressiveness through realistic training to do so.
Sparring is great for this because it gives you a sense of sincerity. Sparring is not about practicing any kind of rigid drill. In sparring, your partner is actively on the move. The fear of being hit increases.
A training fight is ultimately just training and not an emergency, but you are actively helping to increase your stress level.
And at the latest after you've gotten the first, harder hit, your stress levels will continue to rise.
Over time, increase this stress level. That's the only way to get used to the new challenges, bit by bit.
Hard and realistic training is the key to optimally prepare for the situation of a physical confrontation on the street.