Why is All Might freaked out in the big villian fight scene?
Why wouldn't it be? The bad guy just explained that he has robbed a ton of people from their quirks (quite possibly killing them in the process) and now when All Might himself is at his weakest, and Midoriya has still not mastered his gifted quirk, the danger is tremendous. Either of them or even both could be attacked and their quirk stolen, which would definitely be extremely bad.