Magnetometer coil equalization I have a search coil magnetometer with two coils, one at each end of a ferromagnetic core of high magnetic permeability:

Figure 1. Double search coil magnetometer. (Modified from a drawing CC BY-SA 3.0 Christophe coillot.)

Each coil converts change in the magnetic field into a voltage signal. The two signals are digitized. I'm only interested in local anomalies in the magnetic field as they appear in the difference signal between the two coils when the magnetometer is moved through the anomalies. The anomalies are in Earth's magnetic field, created by ferromagnetic objects buried in the ground. So this is a kind of metal detector.

The problem is that the coils are not aligned perfectly symmetrically and are not built perfectly identically. The difference signal is therefore polluted by imperfectly cancelled noise. The noise originates primarily from accidental shaking of the magnetometer that changes its orientation with respect to Earth's magnetic field.

I can do calibration by going to nature and lifting up the magnetometer in the air, away from stray magnetic fields and magnetic anomalies, and shaking it, producing pure noise that ideally would be identical at both coils, but is not, as described above. During such calibration, I have tried adaptively changing the gain of one of the coil signals to match that of the other, which works, but only to an extent. I think I would get better results by matching the frequency responses as well. How can I do this?

Figure 2. Possible signal flow diagrams with equalization of A) just one of the channels to match the other, B) both channels to match a common target response.

I don't need to flatten the frequency responses, as long as they are equal, because I'm not trying to properly quantify the anomalies, just detect them. As far as I understand, the equivalent circuit of each search coil is:

Figure 3. LTspice equivalent circuit of a search coil. Voltage from the voltage source is proportional to magnetic flux time derivative. The component values are very approximate, but probably close enough to reveal the general shape of the frequency response.

Figure 4. LTspice simulated frequency response of the search coil.

Gain must remain as one of the variables to equalize, because some of the gain difference comes from the digitization. I can set the sampling frequency as high as needed. I'd rather not do the calibration in a test bench with a generated artificial magnetic field, but adaptively in the field.
I would equalize both channels.  The most robust method is to cut both back to a lower bandwidth.  Then you only need to equalize the gains.
Assuming that your scanning by hand a 1-2 second sweep rate seems reasonable.  Unless your imaging, you can narrow the sweep down to smaller distances when a signal needs investigation.
A 10-50Hz bandwidth should accommodate human response times.  In this case you can use simple "RC" filtering to a fixed standard on both samples.
You mention construction variations; so first, you need to buffer yourself away from the coils.  Then apply a fixed RC filter.  Since you have isolated yourself both in time and impedance from the coil variations you have complete control of the balance between the two sides.
Of course, this might all be done in software but make sure your A/D has sufficient resolution (time and amplitude) and compliance.