Will this work? Transistor to keep the uC power supply on once a trigger happens
The basic idea is good, but:

For high side switch use PNP (or P-chan) driven with NPN. With your design you will push to LDO 2.6V only.
The button press must be longer then LDO start-up time + charging filters time + boot time of MCU + time to set pin high. Set MCU fuse to startup as fast as possible and place setPinHigh() code very close to beginning can help you.