Dropout dividing by compensation term = overshoots the result? When applying dropout mask, why is it acceptable to divide the resulting state by the percentage of survived neurons?

I understand that it's to prevent signal from dying out. But I've done the test, and found that it disproportionally magnifies the resulting state.

Assume the original state is $(0.1, 0.1, 0.2, 5.0)$ 
and our mask is $(0, 0, 1, 1)$ (with 50% of neurons that survive).

So, the original length is $$
\begin{align*}\sqrt{(0.1\times 0.1 + 0.1\times 0.1  + 0.2\times 0.2 + 5\times 5)} &amp;= \sqrt{25.06}
\approx 5.006.\end{align*}$$

As for the masked vector, its length is $$\sqrt{0.2\times 0.2 + 5\times 5} = \sqrt{25.04} \approx 5.004.$$

Applying the compensation gives $5.004 / 0.5 = 10.008.$

This seems incorrect: my compensation just blew up the state vector. Perhaps we should be compensating differently - more carefully? I think it would even get worse if we mask the individual weights (like DropConnect does).

In my actual test, the state-vector of $192$ elements has length of  $0.885$ and the masked vector, with compensation has length of $1.305$
Your example is cherry-picked: You mask out small numbers and keep a large one.
But dropout is applied randomly. Each of the following six masks, and of the corresponding values for the vector length, is equally likely to appear:
$$
\begin{align*}
&amp;(1, 1, 0, 0): &amp;\sqrt{0.1^2 + 0.1^2} &amp;\approx 0.1414,\\
&amp;(1, 0, 1, 0): &amp;\sqrt{0.1^2+ 0.2^2} &amp;\approx 0.2236,\\
&amp;(1, 0, 0, 1): &amp;\sqrt{0.1^2+ 5^2} &amp;\approx 5.0010,\\
&amp;(0, 1, 1, 0): &amp;\sqrt{0.1^2 + 0.2^2} &amp;\approx 0.2236,\\
&amp;(0, 1, 0, 1): &amp;\sqrt{0.1^2 + 5^2} &amp;\approx 5.0010,\\
&amp;(0, 0, 1, 1): &amp;\sqrt{0.2^2 + 5^2} &amp;\approx 5.0040.\\
\end{align*}
$$
The average vector length is
$$
\frac16 (0.1414+0.2236+5.0010+0.2236+5.0010+5.0040) = 2.5991,
$$
which is roughly half of the original vector length $5.006$. So it makes sense to divide it by the dropout rate of $50\%$.