Latching circuit latches before pushing the latch button
The PNP, like all transistors, has leakage.  So does the NPN.  Hence, there is a (very very small) current flowing at all times.  Because of the feedback (any leakage thru the PNP hits the NPN's base, is amplified, and sent back to the PNP to drive it on even harder), this circuit turns completely "ON" as soon as power is applied.

There is a very easy fix.  Stick a "large-ish" (try 10k) resistor from the collector of the PNP to ground.  This resistor should pull the leakage thru the PNP down below the point where the base of the NPN starts to turn on, preventing it from feeding back.

A big reason why your circuit does this is that there is nothing to draw the PNP's leakage away.  Although you have an LED and 220Ω in that position, remember the LED is a diode and hence has a very high impedance until it starts to forward bias (typically about 1.5V for red LEDs).

If the 10k I suggest still doesn't fix the issue, take a 2nd 10k and put it across the B-E junction of the PNP.  This will draw any of the NPN's leakage away from the PNP's base, thus (hopefully) fixing that also.  With both the PNP's and NPN's leakages drained away from the precious bases, the transistors should power up in the OFF state, and you should be good to go.

1 more thing, 220Ω is a bit small for a modern LED.  If the PNP goes on fully (which it will), say its V_BE_sat about 0.4V, and 1.5V across the LED, then you have ~ 3.3V across the 220Ω in series with the LED, which is 16.5mA.  For a modern small LED, that's kind of generous (they're quite bright even in 5mA range).  But I am just quibbling...If you want to see your LED 100 feet away...well then ignore what I just said :)