RF transformer design for a capacitance variation sensor
Short story: In my opinion, don't change your basic idea covered in your points (1 to 4)

Both "transformer" circuits you have offered-up appear to use types of double balanced mixers that demodulate the audio from the microphone after the microphone has modified the phase shift of an oscillator. 

The microphone is excited at the XTAL frequency by either a direct winding (Sennheiser schematic) or via a capacitor (1 pF, C5) in the 2nd diagram. The mixer stage is like the traditional balanced mixer using transformers: -



I'm not going to go into detail how each circuit in the question uses a double balanced mixer similar to the traditional one in my picture. If you want more details, you'll have to study a bit more.

Suffice to say that a double balanced mixer has one fundamental property that makes it useful for "mixing" two RF signals and that is signal multiplication.

And what you get with signal multiplication is an output level that can vary significantly with amplitudes i.e. it can be used as a phase detector but you need to keep signal and reference amplitudes stable or you'll get a phase angle DC output level that is also somewhat made erroneous by amplitude variations in the input signals.

So, to use DBM accurately as a phase detector you need to have a limiter circuit or saturate the mixer.

Given that you are operating at (only) 11 MHz there are a few quite fast EXOR gates that can do the job - after all, you have an oscillator already at CMOS levels and the tank circuit (fed via a resistor) from your oscillator can have an output level that is easily amplified to CMOS levels via a fast schmitt trigger (plenty to choose from) so, in my humble opinion, this is the best route to take.


  unlike XOR gates, add no jitter


You are operating at 11 MHz and your base bandwidth might be (say) 50 kHz, so what do you really think will be the problem of jitter. At (say) 50 kHz, there would be 220 cycles of clock jittering a bit this way and that way but if you averaged the effect of the jitter, just how much base band noise is really going to be present?

You can easily simulate this and find out BTW.

Think about all those RF receiver chips that use Gilbert cells to electronically perform multiplication - how much noise do they produce when they demodulate (say) an FM broadcast using quadrature detection? Are these chips all as noisy as hell? No they aren't but, you could say that they are operating at carrier frequencies about ten times higher than 11 MHz so the base band filtering will be ten times better.

But those gilbert cell mixers are dealing with a low level RF signal - a signal that is much much smaller than what appears across your tank. So, what is a gilbert cell?

Its forerunner was an attempt to design an exclusive OR gate which brings us nicely back to my original claim that I really don't think you are going to improve on using an EXOR gate acting as a quadrature detector.

I've /designed built sensitive capacitance probes and the minimum delta signal level was less than 20 femto farads - in other words this change in capacitance could be discerned on the demod output when connected to an oscilloscope.