Do grad students in the USA typically have more or less student debt than undergrads?
I think you are underestimating the number of graduate students who do not get any form of financial support. Like you said, most students in STEM or engineering PhD programs full receive full support. PhD students in humanities/social sciences departments are less funded compared to STEM students, but at least are given tuition remissions. 

On the other hand, majority of MA students in any discipline generally receives no support. Then there is MBA students, law students, doctoral residents etc. They are the ones who make up a big portion of all graduate students and most of them have to get student loan to complete their graduate programs. 

For example, a typical law school graduate would end up with 60k-100k+ loans at the time they pass the bar and get a job.