PNP linear regulator problem
Here is why it doesn't work.  The reason it works if you apply the load after the supply has stabilized is because of the 3.3k resistor across the PNP.  Without the 3.3k it wouldn't work because your 2.5V reference would never come up. With a load applied before the power, it draws all of the power and reference stays at zero.  Remove the 3.3k and get the power for your reference at the PNP emitter.  This will ensure the reference always has power.  You can also replace the 7805 with a 2.5V reference.