recurrent neural networks the following points are to be noted the rnn computation involves computing the hidden state for an entity in the sequence this is denoted by ( the computation of (tuses the corresponding input at entity (tand the previous hidden state ( the computation of (tis done for each entity in the input sequence ( ) ( ) ( the output is computed using only the last hidden state ( while computing the current hidden statea set of weights are associated with the input and the previous hidden this is denoted by and wrespectively there is also bias termdenoted by weights are associated with the hidden state while computing the output this is denoted by there is also bias termdenoted by the tanh activation function is used in the computation of the hidden state the softmax activation function is used in the computation of the output training rnns this section describes how rnns are trained we first need to look at how the rnn looks when we unroll the recurrence relationshipwhich is at the heart of the rnn unrolling the recurrence relationship corresponding to rnn is simply writing out the equations by recursively substituting the value on which recurrence relationship is defined