Explanation of this L2 minimization equation
As in the second line, the first two terms are $\mathbb{E}_{\hat{y}}$, it means the variable of the expectation is $\hat{y}_i$ and you can take out $y_i$s from $\mathbb{E}_{\hat{y}}$s. Now we can use $\mathbb{E}\{\hat{y}_i\} = y_i$ and rewrite the terms of the second line likes the following:
$$
\mathbb{E}_{\hat{y}}(\sum_i y_i)^2 = (\sum_i y_i)^2 \\
\mathbb{E}_{\hat{y}}\left[(\sum_i y_i)(\sum_i \hat{y}_i)\right] = (\sum_i y_i) \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i) = (\sum_i y_i)^2
$$
The second line is written based on linearity of the expectation and $\mathbb{E}\{\hat{y}_i\} = y_i$. Hence, we can rewrite the second line like the following as $\sum_i y_i = \mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)$:
$$
\frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right]
$$
And the final step is using this formula $Var(X) = E(X^2) - (E(X))^2$  and take $X = \sum_i \hat{y}_i$:
$$
\frac{1}{N^2}\left[\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)^2 - \left(\mathbb{E}_{\hat{y}}(\sum_i \hat{y}_i)\right)^2\right] = \frac{1}{N^2} Var(\sum_i \hat{y}_i)
$$
From variance to covariance, you can use this formula: $$Var(\sum_{i=1}^nX_i) = \sum_{i=1}^n\sum_{j=1}^n cov(X_i, X_j)$$