What is a capacitor doing in this circuit?
Doesn't this circuit behave like an integrator with the capacitor in that position?


No. The capacitor is in parallel to a resistor so the circuit behaves like a PT1-Element - a certain type of low-pass filter.

If you removed R2 the circuit would behave like an integrator.


  My doubt is what is the purpose of using a capacitor in the feedback loop of this circuit?


Opamps are not ideal. They may have problems with high frequencies. The circuit might become instable and produce high-frequency oscillations although no input signal is present.

A circuit shown in your schematic will amplify low frequencies but suppress high frequencies (which might cause problems if they were not suppressed).


  How is it contributing to high input impedance?


Not at all.

However maybe circuits with lower input impedance have less problems with oscillations so no capacitor is needed in that case.