Design of efficient digital interpolation filter
Butterworth low-pass filters are not going to work for this, as demonstrated in the following. You'd need to use other types of filters, for example linear-phase finite impulse response (FIR).

The magnitude frequency response of an order $N$ discrete-time Butterworth low-pass filter can be approximated by that of an analog prototype (from Wikipedia, with a small notation change):


  $$G(\omega) = {\frac{1}{\sqrt{1+{\omega}^{2N}}}},$$


with normalized frequency $\omega$. The Butterworth filter magnitude frequency response is strictly decreasing. For a pass-band edge gain of $a = 10^{-0.001/20}$, that is, -0.001 dB, we'd employ a further gain factor of 0.001 dB to have a gain of 0.001 dB, that is, $\frac{1}{a}$ at 0 Hz. With this gain factor the magnitude frequency response is $\frac{1}{a}G(\omega)$. Then the transition band begins at the frequency $\omega_0$ found by solving:

$$\frac{1}{a}G(\omega_0) = a$$
$$\Rightarrow \omega_0 = \left(\frac{1 - a^4}{a^4}\right)^{\textstyle\frac{1}{2N}}$$

Then we plot the magnitude frequency response at the beginning of the stop band (the end of the transition band) at frequency $\omega_1 = \frac{0.55}{0.45}\omega_0$ as function of filter order $N$:

Figure 1. $\frac{1}{a}G(\omega_1)$ as function of $N$.

Based on the graph, -174 dB of stop band corner gain would require a filter order of about 120. I don't think such a high order is feasible for a recursive filter as it will lead to numerical problems. Filter types other than Butterworth will be able to meet the specification with a lower filter order.