How can we prove that an autoassociator network will continue to perform if we zero the diagonal elements of a weight matrix?
As we have an autoassociative network prototype vectors are both input and output vectors. So , we have that :
\begin{equation}
    T = P
\end{equation}

\begin{equation}
    W = PP^T - QI = TP^T - QI = \sum_{q=1}^Q p_q p_q^T - QI
\end{equation}

Applying a prototype vector as input :

\begin{equation}
    \alpha = W \cdot p_k = \sum_{q=1}^Q p_k p_q p_q^T - QIp_k
\end{equation}

Because they are orthogonal we have :
\begin{equation}
    \alpha = p_k(p_k^T\cdot p_k) - Q \cdot I \cdot p_k = p_k (p_k^T\cdot p_k - Q \cdot I ) = p_k (\textbf{R - Q $\cdot$ I})
\end{equation}

where $R-Q\cdot I$ is the length of vectors

So since ,

\begin{equation}
    W \cdot p_k= (R - Q \cdot I) \cdot p _k
\end{equation}
prototype vectors
continue to be eigenvectors of the new weight matrix .

It is often the case that for auto-associative nets, the diagonal weights (those which connect an input component to the corresponding output component) are set to 0. There are papers that say this helps learning. Setting these weights to zero may improve the net's
ability to generalize or may increase the biological
plausibility of the net. In addition, it is necessary if we use iterations (iterative nets) or the delta rule is used