LSTM - divide gradients by number of timesteps IMMEDIATELY or in the end?
Answer to your question is it doesn't matter. The gradient is just a product of Jacobians (because of the chain rule), so it doesn't matter if you multiply the result or the intermediate multiplicand, these give the same result. Or, better yet, is to multiply the function you're differentiating, since $\tfrac{d(\alpha f(x))}{dx} = \alpha f'(x)$. So instead of manipulating the gradients you should just average your loss over whatever you want it to be averaged over.

Note that I didn't assume anything about the function above. It's because these are fairly general basic rules, and they apply to all kinds of functions, RNNs and LSTMs included.



That said, you might not want to actually average your loss (remembers, it's the same as averaging the gradients) over the timesteps. Average over inputs x – yeah, sure. Over timesteps? Not necessarily a good idea.

If you do average over timesteps, then you essentially decrease importance of longer sequences, and thus your model will care less of making more mistakes in them, whereas if you do not average over timesteps (or if they are all of the same length), then every timestep-loss has the same weight, and the model cares about all of them equally.