Purpose of capacitors connected to both inputs of Op-Amp
Typically filters look like this:

Source: https://www.planetanalog.com/improving-the-cmrr-of-instrumentation-amplifiers/#
Which you could count the resistors of the bridge to make your RC filter.
The important thing is to make sure the filter poles are balanced (if they aren't then you don't get the subtraction benefit of the instrumentation amplifier from EMI noise), with a capacitor this can be challenging because it is difficult to find capacitors with low tolerances to match them. I'd use an XY capacitor. A

Source: https://www.johansondielectrics.com/emi-filter-decoupling-capacitors
Another problem with using a wheatstone bridge for the R part of the filter is the R varies, so it would be better to put a small resistor in between the op amp and bridge to form the RC filter like the diagram above.
The R and C make a low pass filter set the pole to what is appropriate frequency range for you analog system. (if Vout needs a bandwidth of 100Hz, then set the RC filter to 100Hz)