What types of gear/supplies are "must-haves" for private pilots?
Get a good headset and a decent mobile device that can run a charting app. I use my Moto4 with Avare.

Download The Airplane Flying Handbook -and- The Pilot's Handbook of Aeronautical Knowledge from the FAA site (both free).  Also download the Airman Certification Standards (ACS) for private pilot airplane.

The only books that I usually recommend people actually purchase are the aircraft manual and a written test study guide from ASA, Jeppesen, Gleim, or King.