Why does touching the capacitor in my buck regulator cause my micrcontroller to start executing code?
Before you do a lot of debugging test all the other soldering places as well. I think maybe somewhere else it could have connection problems.

Maybe put an additional cap of about 100nF after your buck. It's better to have  also one with a small esr. Touching the cap shouldn't reduce the voltage to 2.2V, I guess this could be a problem.

When you haven't found anything in the hw then there's maybe a hold state for power on. Have you unplugged your debugger and programmed the chip? Maybe he is keeping the chip in reset state until your voltage drop.