Closed form solution for a single layer linear perceptron
Yes, there is a closed form solution.

In the most general terms, $WX = Y$ is a linear equation, so it can be solved as $W = X^{-1}Y$. If $X$ has no inverse, using the pseudo-inverse $X^\dagger = X^T(XX^T)^{-1}$ will give the $W$ that minimizes $\sum_i||Wx_i - y_i||^2$.

For practical computation, it's best to compute the pseudo-inverse from a matrix decomposition, rather than multiplying the matrices explicity. The safest option is the singular value decomposition. The wikipedia article details the procedure.

The principle behind this solution used only basic calculus. Calling the error $E$, we set $\frac{\partial E}{\partial w_{ij}} = 0$ and solve for $w_{ij}$. This is detailed in [1, pages 89&ndash;93) and [2], both available online.

Alternative phrases for this problem are multivariate linear regression and affine registration.

[1] Neural Networks for Pattern Recognition, Christopher M. Bishop. 

[2] Fitting affine and orthogonal transformations between two sets of points, Helmuth Späth, 2004