Zener diode with high source impedance
Taking the example of a BZX84C5V1 zener (because that sprang into my head), it regulates at 5.1 volts with a cathode current of between 4.8 and 5.4 mA. With a cathode current of 1 mA, the regulation point might be down at 4.1 volts.

With a much lower cathode current it might start to regulate at 3 volts (maybe 100 uA) and at 10 uA it might start trying to regulate at some voltage well-below 2.5 volts. The data sheet states that at 2 uA it will be naturally sitting at 2 volts so be aware of any zener diode's limitations - they are not perfect on-off devices and will consume hundreds of uA below their zenering voltage.

This then becomes problematic because your reference voltage is 2.5 volts and the zener may still be taking several tens of micro amps at this voltage and ruining the regulator's stability. It might not be a big deal to you but, then again, it might be enough to put you off the idea.

This is why precision zeners are sometimes used but, are they usable as a protection device? No, not really - they can be just as susceptible to problems as any other chip. Maybe use a 15 volt zener or a reverse diode to the Vcc rail. Maybe try a 10k in series with the pin as a form of protection. On this particular occasion, a zener would be well-down the list of options.

Stick with my original advice of spending "time and effort into not losing your ground connections".