Dropout dividing by compensation term = overshoots the result?
Your example is cherry-picked: You mask out small numbers and keep a large one.
But dropout is applied randomly. Each of the following six masks, and of the corresponding values for the vector length, is equally likely to appear:
$$
\begin{align*}
&amp;(1, 1, 0, 0): &amp;\sqrt{0.1^2 + 0.1^2} &amp;\approx 0.1414,\\
&amp;(1, 0, 1, 0): &amp;\sqrt{0.1^2+ 0.2^2} &amp;\approx 0.2236,\\
&amp;(1, 0, 0, 1): &amp;\sqrt{0.1^2+ 5^2} &amp;\approx 5.0010,\\
&amp;(0, 1, 1, 0): &amp;\sqrt{0.1^2 + 0.2^2} &amp;\approx 0.2236,\\
&amp;(0, 1, 0, 1): &amp;\sqrt{0.1^2 + 5^2} &amp;\approx 5.0010,\\
&amp;(0, 0, 1, 1): &amp;\sqrt{0.2^2 + 5^2} &amp;\approx 5.0040.\\
\end{align*}
$$
The average vector length is
$$
\frac16 (0.1414+0.2236+5.0010+0.2236+5.0010+5.0040) = 2.5991,
$$
which is roughly half of the original vector length $5.006$. So it makes sense to divide it by the dropout rate of $50\%$.