What to do when you hear many bad stories about your supervisor on the first day of your PhD?
Based on my experience because I had the same situation before and regret not doing that right decision since earlier time. I do recommend you first to stay for couple of months and try to figure out how this PI behaves towards you, in the same time try to look for another good lab. I am still burning out as since month ago this what happened to me, you can see my post here: Forced to Quit PhD although the hard and proved work

So, I dont want you to be in the same situation, I really appreciate that you are passionate about the subject, however, if you have a poor supervisor S(he) is going to ruin every thing.

Another advice and really important which I regret not doing it is asking the current students and alumni, don't be lured by citation and their prestige and trust your gut.

I hope you can take the right decision and for sure I don't recommend you to leave, however stay, work hard as much as you can and if you have noticed red flags from s(he), you can switch from the lab.