   Oja's Learning Rule
  I have written a SOM net to classify data.  I am training about 200-400 patterns of about 8-12 inputs.  I am getting ok results when training (about 60-65%.) but I have noticed that the weights can diverge quite a bit effecting my results, therefore I want to implement Oja's Learning Rule in the training to limit the weights.  The rule is described here... http://www.scholarpedia.org/article/Oja_learning_ruleI have a matrix library that can solve the PCA in my code but I don't follow where and for what the PCA is applied. The vector notation is foreign to me.Can someone describe this procedure in steps or psuedo code?   
  I think all you need is described at thelink you posted but I will try to give yousome step-by-step description.At first some definition (for more clarity I use the terms of the article):x - names the inputvalue of another neuronw - is the weight of this inputdw - (delta w in the article) measures the growth of the connectionAs there are several input-connections they are representedin a vector/array with the index variable i, so Wi, Xi and dWidefine the weight, inputvalue and weightchange of the i-th connection.y - is the collected input of the neurona - is the learning rateNow we can start with the eqations.1. Sum up y for this neuron:In vector arithmetics:y=wT*x (wT is the transposed vector w) means in scalar arithetics:y=W1*X1 + W2*X2 + ... + Wn*Xnassuming you have n connections to the current neuron.2. Get the weightchange of the i-th connection:Again first vector style:dw = a * (y*x - y*y*w) = a*y * (x - y*w)and scalar splitting up into the i-th component:dWi = a * (y*Xi - y*y*Wi)   = a*y * (Xi - y*Wi)for example the 2nd connections weightchange would be:dW2 = a * (y*X2 - y*y*W2)   = a*y * (X2 - y*W2)3. Add the weightchange to the weight of the connection:vector:w = w + dwscalar:Wi = Wi + dWiThese three steps have to be done for all neurons.Thats all to it. I hope this helped as I am not quite sure if it is what you wanted.   ;  Spynacker,Thank you for your response it is very helpful.  So I don't need any linear algebra/matrix math to solve for weights. But instead if I solve for weights in the steps described in the article (clarified by you), what Oja's Learning rule states is that if the weight vectors converge then the output of the neuron is the PCA of the input."This is the incremental change for just one input vector x. When the algorithm is run for a long time, changing the input vector at every step, one can look at the average behavior. An especially interesting question is what is the value of the weights when the average change in the weight is zero. This is the point of convergence of the algorithm.Averaging the right hand side over the x, conditional on w staying constant, and setting this to zero gives the following equation for the weight vector at the point of convergence:    Cw-w^TCww = 0where the matrix C is the average of xx^T. Assuming the input vectors have zero means, this is in fact the well-known covariance matrix of the inputs.Considering that the quadratic form w^TCw is a scalar, this equation clearly is the eigenvalue-eigenvector equation for the covariance matrix C. This analysis shows that if the weights converge in the Oja learning rule, then the weight vector becomes one of the eigenvectors of the input covariance matrix, and the output of the neuron becomes the corresponding principal component. Principal components are defined as the inner products between the eigenvectors and the input vectors. For this reason, the simple neuron learning by the Oja rule becomes a principal component analyzer (PCA)."Is this correct?   ;  I hope I got you right because my math english is not the bestbut again I will try.It took me some time to understand your quote and I would liketo know the source of this quote, can you post it?Now for the content and your question I must say:It depends on your situation if the description in the quote isright. Taken for itself it seems to be right.If you want to apply this to your problem you have to bear in mindthat the quote assumes your input vector has only components near tozero so that the vectors impact on the whole equation is negligible.Under this condition the goal of the post is to determine by theweigt-vector wether the neural net will converge to some certain state.When the system has reached this state the weight change dW will bezero in average, thus will stay there as no change is applied.Thats enough for now. I hope this post covered your question well.