Coherent averaging of polyphase components of a signal I want to subsample a multi-band signal to introduce controlled aliasing. My frequency components are uniformly distributed across the spectrum (comb structure) and occupy a relatively narrow bandwidth each. The spectrum can be said to be sparse.

I achieve decimation through picking every $n$-th sample from the original signal. This of course does not take advantage of the remaining samples, which are simply rejected. By adding a phase (sample) shift in my decimation routine, I can get $n$ under-sampled signals in total, which are known as polyphase components. I can also incoherently average them (periodogram averaging) in the spectral domain, since the spectrum is of my interest rather then the time-domain representation.

Is it possible to coherently (in phase) average such undersampled polyphase components of my signal? What phase shift should be applied after down-sampling with a factor $n$? In other words, can we simultaneously undersample (decimate) and oversample (average) to take advantage of the remaining samples for quantization noise suppression? 

A typical scenario would be sampling a 1 GHz wide comb signal with a spacing of 10 MHz between the teeth, which we can decimate 10 times to have a spacing of 1 MHz and a bandwidth of 100 MHz. If we sample the signal with an 8-bit analog-to-digital converter, can we use the remaining 9 polyphase components to enhance the vertical resolution of the signal after subsampling?

Thank you,
Lukasz S.
Yes. Simply sum all the polyphase outputs and the sum result will have higher resolution.

Consider that each polyphase output is a delayed version of the same signal, so that if you commutated through all the outputs, you would get a higher sampled version of your same signal and the quantization noise of this signal would be approximately white across this higher digital frequency band. If you passed this signal through a low pass filter prior to taking every nth sample in a decimation process you would indeed benefit by an increased dynamic range to the extent the quantization noise components in each sample are independent. (they should be; the delayed replicas at each output would have independent quantization noise and coherent signal components). So the decimation filter could be a simple moving average in which case it is easy to see that your best solution is to sum average the outputs of your polyphase filters and use that average as the final result rather than just selecting any one of them.  The moving average introduces passband droop- so with more complication you could do more elaborate filtering. I would be inclined to use a CIC filtering approach as the moving average followed by a simple 3 tap inverse sinc shaper if I was concerned about the resulting passband droop.