Will adding a pole in the loop gain make the system more unstable?
If you are talking about C1: It will improve stability. Judging from the freqencies that are supplied to the opamp non-ionverting input with the intent of modulating the diode current, the components C1 + R4 are a lowpass. It will limit the maximum frequency the diode can be modulated with.
But for any signal that comes from the opamp output and travels to its inverting input (i.e. the feedback path), the same components are actually a highpass.
If it were missing, the feedback would be very inefficient at high frequencies: The source of Q1 will react with a tiny delay to voltages applied to its gate. This can make the opamp oscillate. If you add a small C1, you damp this oscillation by bypassing the feedback circuit at high frequencies and forming a simple voltage follower at high frequencies. R4 is then placed to decouple the source of Q1 and inverting input at high frequencies but maintain the low frequency feedback connection. The R4 + C1 also define the corner frequency of the filter.
You can obtain the same corner frequency for different RC combination with the same RC product. The tradeoff is between opamp current and noise. Small C and high R (as done in your circuit) will allow not so much high frequency current but will create around 10 nV/rtHz of noise in the feedback path. So for it would be not a good choice if noise performance is paramount. Larger C and smaller R will reduce noise, but increase the output current of the opamp at high frequencies.