Compare the quality of IQ data from different Software Defined Radios (SDRs)
First and foremost, I would recommend against over the air testing for this given the significant challenge in really being able to provide the same signal to each radio (since you have both temporal and spatial constraints that you cannot simultaneously meet). 

I would instead use one GNU radio as a transmitter (or any other repeatable high quality source) to provide a consistent and repeatable waveform that can be used for each of the radios under test with a cabled test where the attenuation can be carefully and repeatably controlled over a wide range.

With this, there are several tests that immediately come to mind:

An ideal test waveform would be band limited complex Gaussian noise (pseudo-random noise as a repeatable pattern) since it will best occupy all possible positions on the complex IQ plane as well as all frequencies in the band of interest. This would then test the radio for a wide range of modulations, but if you have specific modulations in mind then those such waveforms for test purposes would also be ideal. (OFDM waveforms are well represented as complex Gaussian distributed bandlimited waveforms by the way, but it would be sufficient enough to simply create a complex random waveform and band-limit it for this generic case.)

An ideal test metric would be the correlation coefficient, $\rho$, as it will show errors from all sources including noise figure, quadrature error, LO phase noise, ADC quantization noise, IP3, IP2, etc.  For more information on that see Noise detection and How can I find SNR, PEAQ, and ODG values by comparing two audios?

The test should be done over a wide range of input power levels to show the related sensitivity and maximum power handling for comparative receivers. Thus the test metric for all receivers would be $\rho$ vs $P_{in}$.

The $\rho$ may be limited by the transmitter if that used is not of sufficient quality, so care must be taken to confirm that the $\rho$ for the transmit waveform is better than that which would result for all the tests performed (or represent the ceiling of the measurement). For the same reason start with the highest test power with known confirmed $\rho$ and adjust the power level in the cabled test receiver using an inline passive variable attenuator (with a rated much higher input P1dB) since any active device or variable amplifier is more likely to introduce additional distortion.

Additional tests involve how the receiver rejects interference such as intentional jammers or co-site transmitters. This is usually done with one tone testing in an adjacent band to help distinguish those receivers with better LO phase noise, and two tone interference testing in such a way to test for IP3 (and IP2 for zero-IF receivers in particular). The same test as above with the correlation coefficient vs input power is repeated, but in the presence of the jammers which can be done with unmodulated tones. Different recievers can have different sensitivities based on their architecture (or mis-managed spurs from a poor design), so a robust test would sweep the test jammers over a wide frequency range. If the radios are intended for specific applications, then the interference tests can be limited to cases appropriate for their known environment.

Instantaneous dynamic range in band would also be of interest.  As a simple test for this an in band single tone could be used at maximum input and the noise floor and spurious free dynamic range could be assessed in an FFT of the captured IQ signal. (You could also do specific linearity and single-tone and two-tone in band testing if you wanted to isolate these characteristics from what was otherwise in the catch-all $rho$ test described above).