Attempt to reuse RNNCell with a different variable scope I'm using Tensorflow 1.1.0 with GPU support and I have this function:

def get_init_cell(batch_size, rnn_size, keep_prob=0.75, layers=2):
    """
    Create an RNN Cell and initialize it.
    :param batch_size: Size of batches
    :param rnn_size: Size of RNNs
    :return: Tuple (cell, initialize state)
    """
    # Basic LSTM cell
    # lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)
    lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size, forget_bias=0.0, state_is_tuple=True, reuse=tf.get_variable_scope().reuse)

    # Add drop to the cell
    drop = tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)

    # Stack multiple LSTM layers
    rnn_cell = tf.contrib.rnn.MultiRNNCell([drop for _ in range(layers)], state_is_tuple=True)

    # Getting an initial state of zeros
    initial_state = rnn_cell.zero_state(batch_size, tf.float32)

    # Set the name
    initial_state = tf.identity(initial_state, 'initial_state')

    return rnn_cell, initial_state


The previous code works perfectly with Tensorflow 1.0.0 if I use lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size) but now I have changed the way I get the lstm cell. And I get this error:


  ValueError: Attempt to reuse RNNCell
   with a different variable scope than its
  first use.  First use of cell was with scope
  'rnn/multi_rnn_cell/cell_0/basic_lstm_cell', this attempt is with
  scope 'rnn/multi_rnn_cell/cell_1/basic_lstm_cell'.  Please create a
  new instance of the cell if you would like it to use a different set
  of weights.  If before you were using:
  MultiRNNCell([BasicLSTMCell(...)] * num_layers), change to:
  MultiRNNCell([BasicLSTMCell(...) for _ in range(num_layers)]).  If
  before you were using the same cell instance as both the forward and
  reverse cell of a bidirectional RNN, simply create two instances (one
  for forward, one for reverse).  In May 2017, we will start
  transitioning this cell's behavior to use existing stored weights, if
  any, when it is called with scope=None (which can lead to silent model
  degradation, so this error will remain until then.)


I have changed get_init_cell with this one:

def lstm_cell(rnn_size, keep_prob=0.75):
    # Basic LSTM cell
    lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size, reuse=tf.get_variable_scope().reuse)

    # Add drop to the cell
    return tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)

def get_init_cell(batch_size, rnn_size, keep_prob=0.75, layers=2):
    """
    Create an RNN Cell and initialize it.
    :param batch_size: Size of batches
    :param rnn_size: Size of RNNs
    :return: Tuple (cell, initialize state)
    """    
    # Stack multiple LSTM layers
    rnn_cell = tf.contrib.rnn.MultiRNNCell([lstm_cell(rnn_size, keep_prob) for _ in range(layers)])

    # Getting an initial state of zeros
    initial_state = rnn_cell.zero_state(batch_size, tf.float32)

    # Set the name
    initial_state = tf.identity(initial_state, 'initial_state')

    return rnn_cell, initial_state


And I still getting the same error.

I have also try it with the same error:

def get_init_cell(batch_size, rnn_size, keep_prob=0.75, layers=2):
    """
    Create an RNN Cell and initialize it.
    :param batch_size: Size of batches
    :param rnn_size: Size of RNNs
    :return: Tuple (cell, initialize state)
    """    

    def lstm_cell():
        # Basic LSTM cell
        return tf.contrib.rnn.BasicLSTMCell(rnn_size, reuse=tf.get_variable_scope().reuse)        

    def attn_cell():
        return tf.contrib.rnn.DropoutWrapper(lstm_cell(), output_keep_prob=keep_prob)

    # Stack multiple LSTM layers
    rnn_cell = tf.contrib.rnn.MultiRNNCell([attn_cell() for _ in range(layers)], state_is_tuple=True)

    # Getting an initial state of zeros
    initial_state = rnn_cell.zero_state(batch_size, tf.float32)

    # Set the name
    initial_state = tf.identity(initial_state, 'initial_state')

    return rnn_cell, initial_state


Any idea?
It looks like you might be trying to use MultiRNNCell as a recurrent neural network. This isn't really its purpose; that is what dynamic_rnn, static_rnn, and raw_rnn are for. MultiRNNCell is intended for creating a feedforward, multilayer network which is the "cell" to be iterated to produce a recurrent neural network.

The old version of your code, if I understand it correctly, creates a network consisting of a sequence of LSTM layers, each with its own set of weights. If your goal is to make the weights shared between layers (so that your network is recurrent), you want to instantiate a dynamic_rnn and pass it a single instance of one of the RNNCell subclasses. That instance generates all of the layers of the network by having its __call__ method invoked repeatedly, once for each layer. The dynamic_rnn instance handles the scoping and variable reuse issues for you and connects the layers together.

You have at least three options for the cell to pass to the dynamic_rnn constructor:

1) A DropoutWrapper which was created using a BasicLSTMCell (essentially, the output of one call to attn_cell, but get rid of that reuse option). Do this option.

2) A MultiRNNCell consisting of an LSTM layer, followed by a dropout layer (unneeded here, but this is what you would do in some more general case).

3) Subclass rnn_cell and create your own type of layer.