Why is the variance going down so much in this weight initialization problem(using pytorch)?
Basically, because the variance is quadratic in w.

You can consider your problem as computing the variance over scalar products $z = \vec{x}\vec{w}$ by drawing 500000 samples of $\vec{w}$ from $p(\vec{w})$. Let's compute the mean first

$$ E_{p(\vec{w})} [ \vec{w}^T\vec{x} ] =  \vec{\mu}_w^T \vec{x} = 0$$

because you draw the $\vec{w}$ from a zero-centered Gaussian. With the expectation value as zero, the variance is given by

$$ E_{p(\vec{w})} [ (\vec{w}^T\vec{x})^2 ] \\
= E_{p(\vec{w})} [ \vec{x}^T\vec{w} \vec{w}^T \vec{x} ] \\
= \vec{x}^T E_{p(\vec{w})} [ \vec{w} \vec{w}^T ] \vec{x}
$$

$E_{p(\vec{w})} [ \vec{w} \vec{w}^T ]$ is the covariance matrix of the $\vec{w}$ and since they stem from independent standard normal distributions is diagonal with $\sigma_w^2 = 1$ on the diagonal. So that gives

$$
E_{p(\vec{w})} [ (\vec{w}^T\vec{x})^2 ] = \vec{x}^T \vec{x} \approx 512
$$

as you are approximating the sum of variances of 512 independent samples of standard Gaussian. If you now scale the $\vec{w}$ with a constant $k$, you can do this calculation again and it will give you

$$
E_{p(\vec{w})} [ (\frac{1}{k}\vec{w}^T\vec{x})^2 ] \\
= \frac{1}{k^2} E_{p(\vec{w})} [ (\vec{w}^T\vec{x})^2 ] \\
\approx \frac{1}{k^2} 512
$$

So in order to end up with 1 in the very end, you need to scale with $k=\sqrt{512}$.

I hope this answers your question.