Is an ungraded pass going to affect my PhD applications?
I have doubts that it will matter. The reasons for taking a pass might be obvious to a reader. Letting the numbers fall where they may might seem more honest.
But also consider that few (if any) programs will automatically exclude you for GPA alone. People are admitted into doctoral programs based on many things, none of which (other than dishonesty, perhaps) are determinative alone. People on an admissions committee will want to make a prediction about your likely success in the program and thereafter as well as your readiness to do research.
I also suspect that most committees will be a bit generous in the age of coronavirus, though that doesn't lessen the competition that every student faces. Make your best case for admission no matter which way you choose to jump on this (minor) issue.