Dithering adds harmonic distortion after filtering and upsampling
It seems like this is an expected behavior because -96 dBFS signals are too much correlated with the quanitzation error due to 18 bit trancation.

Adding more dither, like on 2 LSB does in fact get rid of those extra harmonics. However, the noise floor is getting much larger than expected. I don't think it's possible to have the best SNR in audible range while shaping the noise away to the end of the spectrum. I've tried that with 2nd noise shaper (TDPF dither with high-pass function) and it did not give any good results.