Why don't we gradually update the activation parameters in RNN from one activation to the next as the network is learning more?
The motivation to use RNN is that the length of the sequence or position info is random in the data. 

For example we could use the sample trained RNN model to translate the following setences:


I am a man
I am a woman and you are a man


In RNN, we do not consider the position of the words, only consider the relation between words. Thus the different activation parameters trained for different positions are useless. 

Moreover to make RNN better informed (using previous words / next words) we could use Gated recurrent unit (or LSTM) and Bidirectional RNN.