How does this slow turn-on regulator work?
Initially, the capacitor voltage is zero and the PNP transistor is on, shorting out R2 and fixing the output at ~1.25V (see the datasheet on how R1/R2 set the output voltage). Since there is still some output voltage, it will slowly charge the base-collector capacitor through the 50k resistor, bringing the transistor base-emitter voltage down until it finally shuts off completely. Now R2 is pretty much the only element in the feedback loop, giving the desired output.

The diode is there to protect the regulator from cases where the output is shorted. It's described in the "protection diodes" section of the datasheet.

It's an interesting little circuit.