Gradient Checking LSTM - how to get change in Cost across timesteps?
Interesting question.


  Like standard backpropagation, [backpropagation through time] consists of a repeated application of the chain rule. The subtlety is that, for recurrent networks, the loss function depends on the activation of the hidden layer not only through its influence on the output layer, but also through its influence on the hidden layer at the next timestep.


It looks like both approaches would have similar results but at a different granularity (adding noise at different levels) this is because the backprop is not really disturbed by this addition as it is still chained from last time step to the first. Hence it boils down to why you actually want to add noise in the first place, as mentioned in this paper , sections IIIC and IV.