@register("<STR_LIT>")<EOL>def lstm(nlstm=<NUM_LIT>, layer_norm=False):
def network_fn(X, nenv=<NUM_LIT:1>):<EOL><INDENT>nbatch = X.shape[<NUM_LIT:0>]<EOL>nsteps = nbatch // nenv<EOL>h = tf.layers.flatten(X)<EOL>M = tf.placeholder(tf.float32, [nbatch]) <EOL>S = tf.placeholder(tf.float32, [nenv, <NUM_LIT:2>*nlstm]) <EOL>xs = batch_to_seq(h, nenv, nsteps)<EOL>ms = batch_to_seq(M, nenv, nsteps)<EOL>if layer_norm:<EOL><INDENT>h5, snew = utils.lnlstm(xs, ms, S, scope='<STR_LIT>', nh=nlstm)<EOL><DEDENT>else:<EOL><INDENT>h5, snew = utils.lstm(xs, ms, S, scope='<STR_LIT>', nh=nlstm)<EOL><DEDENT>h = seq_to_batch(h5)<EOL>initial_state = np.zeros(S.shape.as_list(), dtype=float)<EOL>return h, {'<STR_LIT:S>':S, '<STR_LIT:M>':M, '<STR_LIT:state>':snew, '<STR_LIT>':initial_state}<EOL><DEDENT>return network_fn<EOL>
Builds LSTM (Long-Short Term Memory) network to be used in a policy.
Note that the resulting function returns not only the output of the LSTM
(i.e. hidden state of lstm for each step in the sequence), but also a dictionary
with auxiliary tensors to be set as policy attributes.

Specifically,
    S is a placeholder to feed current state (LSTM state has to be managed outside policy)
    M is a placeholder for the mask (used to mask out observations after the end of the episode, but can be used for other purposes too)
    initial_state is a numpy array containing initial lstm state (usually zeros)
    state is the output LSTM state (to be fed into S at the next call)


An example of usage of lstm-based policy can be found here: common/tests/test_doc_examples.py/test_lstm_example

Parameters:
----------

nlstm: int          LSTM hidden state size

layer_norm: bool    if True, layer-normalized version of LSTM is used

Returns:
-------

function that builds LSTM with a given input tensor / placeholder