How can I best determine if a masters program is of high quality?
There are several things you could try


Read student reviews online - sites like GradCafe offer
interesting discussions that will give you an idea how well the programs are run and how much faculty members are willing to go the extra mile to help their students get jobs after they graduate.
Look at USNews ranking as a loose reference. In my opinion (as a mathematician) those rankings do not fully reflect how good an institution is, but a department ranked #1 is quite likely to be better than one ranked #100. Bear in mind that US News won't give you the full picture, though, you could run into a fantastic professor at an institution that only gets a decent ranking.
Speak to the academic advisor in your department


Last but not least, your choice very much depends if you want to stay in the academia. If you're going into industry, reputation and the alumni network might be factors to consider. Otherwise, browse through the faculty directories and see if there's any potential match of research interests. For example, if you want to learn analysis, you're better off going to UCLA than to Harvard.

Good luck!