Gradient Checking LSTM - how to get change in Cost across timesteps? I am performing gradient check for my LSTM which has 4 timesteps. The LSTM looks as follows:

   01       01       01       01
   ^        ^        ^        ^
  LSTM --&gt; LSTM --&gt; LSTM --&gt; LSTM
   ^        ^        ^        ^
   11       11       11       11


So, at every timestep we are feeding in vector {1,1} and expect {0,1} at the output.

Assume I perturb the weight inside LSTM, then perform 4 forward props - one for each timestep - how do I now get delta of the cost function that this single perturbation has caused?

Am I allowed to simply add-up the change in Cost from all 4 timesteps to treat it as derivative estimate?



Also, should I perform it as follows for LSTM:


perturb a single weight upwards
forward prop 4 timesteps
perturb the weight downwards
forward prop 4 timesteps
get 4 deltas
sum the 4 deltas to get a total change in Cost


or


Set N=0
perturb the weight upwards
foward prop at a particular timestep N
perturb the weight downwards
forward prop at a particular timestep N
get single delta, store it away
increment N
until N not equal 4 return to step 2)
sum the 4 deltas to get a total change in Cost


The second approach somehow seems more correct, because LSTM will have a hidden state ..Is this correct intuition or it won't matter?
Interesting question.


  Like standard backpropagation, [backpropagation through time] consists of a repeated application of the chain rule. The subtlety is that, for recurrent networks, the loss function depends on the activation of the hidden layer not only through its influence on the output layer, but also through its influence on the hidden layer at the next timestep.


It looks like both approaches would have similar results but at a different granularity (adding noise at different levels) this is because the backprop is not really disturbed by this addition as it is still chained from last time step to the first. Hence it boils down to why you actually want to add noise in the first place, as mentioned in this paper , sections IIIC and IV.