Decimate ADAR7251 Sigma-Delta ADC Output
The difference between taking every 4th sample, and cascading two half-band decimation filters, is one of noise aliasing.

In the first case, you resample all the noise in the wide bandwidth, and fold it into your signal spectrum.

In the second case, you remove the noise in the wide bandwidth, leaving only the noise that's already within the signal spectrum.

Where the noise has a flat spectrum, for instance with a typical SAR ADC, doing it the simple way leads to a degradation of 6dB in the signal to noise ratio (SNR).

In a sigma delta ADC, the noise tends not to be flat, but rises towards higher frequencies. You would have to study the specifications for your particular ADC, to see how much rising noise the designers have allowed through the decimation filters at any particular decimation setting. Doing it the 'easy' way could result in a much more than 6dB degradation in SNR. 

The worst place from your point of view is noise around the Nyquist frequency, as this will fold down to the baseband. Unfortunately, this is likely to be where the ADC designers have eased up on the noise filtering, as it's a) harder to filter here and b) a long way from the passband, so likely to be of no consequence to a user at that final sample rate.

You will know from your application what SNR you need. It may be that the ADC provides enough margin for you to be able to tolerate 6dB or more degradation. If you can't, then you need to filter while you decimate. Note that a noise filter does not require much in the way of a stopband, so your half-band filters do not need a deep stopband. Unfortunately, in a halfband filter, the stopband depth is intimately related to the passband flatness, so you may need a longer filter anyway if flatness is an issue. It may be better to design a 4:1 decimation filter with independent control of passband and stopband, rather than kludge it with two halfband filters. Only detailed design with your specifications would tell.