low noise, low distortion analog multiplexing
One aspect you haven't considered is that with an inverting mixer, the mixing node is a virtual earth hence, you "mix" input currents and each input's current "sinks" into a virtual earth. This provides one major benefit: -

Very little cross talk between one input signal and another.


In other words, an input signal hardly\$^1\$ gets its signal-current messed-with from other inputs signals. This does not happen on the non-inverting op-amp mixer because the signal levels depend on each other and the source impedances of other signals connected this way. This then leaves U1 or U2 as the main contenders: -



In a mixer like this, the mixing-node suffers a lot from all the inputs being connected to it so I would go for the circuit that uses U1. Yes, there will be more capacitance to ground at the mixing-node and this will cause high frequency noise but so will having a bunch of inputs and that is a problem faced by all anlogue mixers so, choose an op-amp with low input noise voltage density and be prepared to add a parallel capacitor across Rf.

You must also remember that at high audio frequencies, analogue switches are not open circuits and some high spectrum noise from an input that is deemed to be turned-off may still be heard.



\$^1\$ I used the word "hardly" because with an op-amp there is finite (and not infinite) gain and the virtual earth summing point becomes a slight abstraction. This means that the virtual earth can be at a few mV p-p and at higher frequencies (where the op-amp open-loop gain reduces) it might be 10 mVp-p for example. It's still a lot better than the non-inverting summing node of course.