AC->DC Power supply ground and probing
Why is the DC ground connected to mains earth?  While I am not overly familiar with safety qualifications, I believe it is now mandatory, and has been "best practice" for quite some time, to have some potential in the system tied to earth, and typically to the chassis.  As long as the DC output is not required to be floating, the most common connection is earth/chassis/DC ground all tied together.  This output is generally the one of use to the human being utilizing the supply.  This is different than say a lab/bench supply where you typically have a "-" prong that is the floating supply's return path and a GND/Earth prong that is tied to chassis and earth for safety reasons.  This allows you to stack supplies which you can't do in the case of a earth-referenced output.

As for the LED, you are exactly correct.  But, if chassis = DC GND = earth, this works.  Whether or not that is allowable by modern-day safety standards is unknown to me.  This isn't that much different than how an automobile is typically wired, and I'm sure that wasn't completely uncommon "back in the day".  I believe that CRT TVs/monitors still had live chassis connections even into the 90s.

I don't feel very comfortable answering the last part of your question.  If this is an isolated supply with earth connection on the DC/output side, the AC side has to be considered completely floating.  Don't go sticking a probe in there with your ground clip on the DC side.  Tektronix did a very good whitepaper on how to probe floating supplies several years back.  I highly recommend you try to track that down.  I believe it was called Fundamentals of Floating Supply Measurement.  Be safe!