what is the correct order of applying a nonlinear operation and downsampling (with anti-aliasing)
When one nonlinear and one linear operation are suitably adapted to the problem, the nonlinear one is quite often applied first. Nonlinear processing is often applied to modify data such that more classical algorithms apply more easily, even when the original data does not follow the proper assumptions. Examples are:

removing outliers/trimming data before computing averages,
applying variance stabilizing transformations (Anscombe) to use regression needing constant variance,
computing a logarithm to convert a multiplicative problem before using an additive technique on the log-transformed data.

Of course, applying a non-adapted nonlinear transform first can be very harmful. Yet in you case, a log-coompression may either:

reduce the impact of extreme data values
stabilize an heteroskedastic noise
account for multiplicative effects

before an interpolation (a kind of weighting average or filtering). This goes under the hood of homomorphic filtering.