Does the paper "On the difficulty of training Recurrent Neural Networks" (2013) assume, falsely, that spectral radii are $\ge$ square matrix norms?
It is an error. But it is also not the in final version of the paper (arxiv.) The final version of the paper can be found here where they replace "absolute value of the largest eigenvalue" with "largest singular value". 


  We first prove that it is sufficient for $\lambda_1 &lt; \frac{1}{\gamma}$, where $\lambda_1$ is the largest singular value of $W_{rec}$, for the vanishing gradient problem to occur.