deep learning for text and sequences output - - output compute new carry ct ct state compute new carry output_t activationwo*input_t uo*state_t vo*c_t boinput - figure output + input + carry track ct state + input + anatomy of an lstm if you want to get philosophicalyou can interpret what each of these operations is meant to do for instanceyou can say that multiplying c_t and f_t is way to deliberately forget irrelevant information in the carry dataflow meanwhilei_t and k_t provide information about the presentupdating the carry track with new information but at the end of the daythese interpretations don' mean muchbecause what these operations actually do is determined by the contents of the weights parameterizing themand the weights are learned in an end-to-end fashionstarting over with each training roundmaking it impossible to credit this or that operation with specific purpose the specification of an rnn cell (as just describeddetermines your hypothesis space--the space in which you'll search for good model configuration during training--but it doesn' determine what the cell doesthat is up to the cell weights the same cell with different weights can be doing very different things so the combination of operations making up an rnn cell is better interpreted as set of constraints on your searchnot as design in an engineering sense to researcherit seems that the choice of such constraints--the question of how to implement rnn cells--is better left to optimization algorithms (like genetic algorithms or reinforcement learning processesthan to human engineers and in the futurethat' how we'll build networks in summaryyou don' need to understand anything about the specific architecture of an lstm cellas humanit shouldn' be your job to understand it just keep in mind what the lstm cell is meant to doallow past information to be reinjected at later timethus fighting the vanishing-gradient problem concrete lstm example in keras now let' switch to more practical concernsyou'll set up model using an lstm layer and train it on the imdb data (see figures and the network is similar to the one with simplernn that was just presented you only specify the output dimensionality of the lstm layerleave every other argument (there are manyat the keras licensed to