Problem with a tracking LM337 stability
I solved the problems with the help of a friend,
So the solution to the instability was because of the configuration of the integrator and also the capacitor which was needed for the ADJ pin.
I reduced the R4 value from 1K to 470 ohm and then placed C1 between ADJ pin and non-inverting input of the op-amp. that solved the oscillation problem with and without load.
For the second problem I added a buffer stage to the reference voltage, so Q1 only have to pull down the reference voltage which will draw way less current.