Must graphs in a mathematical PhD thesis be drawn nicely?
Even if it's not a formal requirement, consider how important first impressions are. People will judge a paper with sloppy diagrams, and assume that the results and writing will also be sloppy. This is especially true for a PhD thesis. Since you've had 4-5 years to work on it, people will assume that a thesis is well-written and every aspect carefully planned. That includes the figures. A thesis with hand-drawn figures will not be seen as professional.

I personally recommend Inkscape for drawing things. I think it's easier than Illustrator. I believe for my undergraduate thesis I picked up most of the knowledge i needed in 2 hours of watching Youtube tutorials and trying them out. Fortunately, math diagrams usually only require knowledge of the most basic tools in a program.

If you are truly desperate, I believe it is usually acceptable to pay someone to take your pencil drawings and turn them into professional ones, as long as you give proper credit (check your university guidelines though). There is probably an undergrad at your university with experience in Inkscape or Illustrator who wants to make some extra money.