recurrent neural networks figure - rnn (recurrence using the previous hidden statefigure - also depicts loss function associated with each output associated with each input we will refer back to it when discuss how rnns are trained it is essential to internalize how an rnn is different from all the feedforward neural networks (including convolutional networkswe discussed earlier the key difference is the hidden statewhich represents summary of the entities seen in the past (for the same sequenceignoring for the time being how rnns are trainedit should be clear how trained rnn could be used for given sequence of inputsan rnn would produce an output for each entity in the input let' now consider variation in the rnn where instead of the recurrence using the hidden statewe have recurrence using the output produced in the previous state (figure -