Reducing Undershoot and Overshoot in a custom voltage regulator?
I only read your first paragraph and looked at the schematic, but there are some obvious problems already:


Get rid of C1.  That slows down the plant response, which is always a bad thing for stability.  Worse, its effect is asymmetric with rise and fall.
Get rid of C2.  This is slowing down the feedback signal, which is a really bad thing to do to any control loop.
Add a little resistance from the base of Q2 to ground.  This will help the symmetry of the response a little, and will make the plant more linear at low voltages.  It will also overcome leakage current thru Q1 to keep the system fully off when the base of Q1 is driven low.  I don't know what currents this is intended for, but 1 k&Omega; to 10 k&Omega; should work.
Add a little compensation immediately around the opamp.  Put a small cap directly between the opamp output and its negative input.  Then put some resistance in series with the output voltage feedback signal into the opamp negative input.  Start with 100 pF for the cap and 10 k&Omega; in series with the feedback to see what happens.  Adjust the feedback series resistor to get the step response you want.  Lower values will make a faster step response but will cause overshoot.  Higher values will decrease and then eliminate overshoot but at the expense of response time.