Capacitive dropper power supply for LED - zener/decoupling capacitor?
I understand that these work well for driving LEDs since they act like a constant current source. I was wondering if the design could be (safely) simplified further by removing the zener and decoupling capacitor

There are plenty of cheap LED lights just like this. They're all unusable for anything except indicator lights because they flicker at 120Hz though.

In this design R1 is not technically a protective element—it's just for inrush limiting

R1 should be a fusible resistor, which is cheaper in mass production than a resistor and a fuse. These resistors are designed to fail open just like a fuse.
R2 should be specified to withstand the voltage, which probably means several resistors in series.

mains is subject to voltage transients from things like lightning and inductive loads switching off, and that C2 would shunt the majority of these transients away, protecting the LEDs - that seems to make sense, I suppose C2 + the input impedance of the circuit would behave like an RC filter during a transient?

If the transient is fast enough, that depends mostly on C2's ESR. LEDs can also tolerate surprisingly high non-repetitive peak currents.

If that's the case, what purpose does the zener serve if voltage regulation isn't important

For LED lights, I don't see any purpose for it.
When a component absorbs a transient, the transient energy is converted into heat. So its ability to absorb a transient's energy comes from thermal mass of the part of the component where dissipation occurs. A tiny semiconductor junction can absorb much less energy than a bulky voltage-dependent resistor, for example. Unless the zener is much bulkier than the LEDs and rated for high current, it won't have much more transient handling capability.