Recurrent Neural Network (RNN) Vanishing gradient problem - Why does it affect earlier timesteps more? I understand the concept of backpropagation in standard neural networks and  backpropagation through time with RNNs, why this causes exponentially smaller gradients at earlier time steps and most of the maths behind it all, but what I don’t understand is why this affects the earlier timestep in particular? Since, the parameters (weights) in the RNN are all shared between timesteps, why is it that the earlier timestep is more affected? Wouldn’t they all be affected since they all share the same badly optimized weight which is never updated due to the many small terms in the multiplicative product which produces $\frac{\partial E}{\partial w} $? I feel like I'm totally misunderstanding something here. Many thanks
Of course that all weights are the same, but the update applied to the weights has a contribution from each of the timesteps, and the contribution associated with the first timesteps is what is more affected by the vanishing gradient problem.