PLL deviation from matched frequency
Hence even if the output frequency is exactly the same as that of input, we may get an error from phase detector and consecutively the VCO will change the output frequency, which further deviates the output from input frequency. 
  ...
  
  What am I missing?


If the PLL control loop is stable, this frequency deviation will force the error signal smaller. As the error signal decreases, the frequency deviation will also be decreased. Eventually, the error signal and the frequency deviation will be smaller than the noise sources in your system, so you won't be concerned about them any longer.

There will always be (hopefully) small frequency deviations due to noise in the control signal, so any hope that the output frequency will be perfectly locked to the input frequency is unfounded.

Also there will be an offset error in a real phase detector, which will lead to the locked condition happening with a slightly non-zero error signal value. But this error signal will be constant (if we ignore noise) once the PLL is locked.