<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>OS Platform: OS X 10.13.3</li>
<li>Custom code</li>
<li>tensorflow version: 1.12.0</li>
<li>python version: 3.6.5</li>
</ul>
<p dir="auto"><strong>Describe the current behavior</strong><br>
Tensorflow raises a TypeError when creating a dynamic_rnn with tf.int32 type in its input and state. When changing the type to tf.float32 the error is not raised.</p>
<p dir="auto"><strong>Describe the expected behavior</strong><br>
Ideally, a dynamic_rnn should support tf.in32 types. If there's any reason why instantiating a dynamic_rnn with tf.int32 type in its input and state should not be allowed, a custom error should be raised.</p>
<p dir="auto"><strong>Code to reproduce the issue</strong><br>
The code below reproduces the error:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow as tf

X = tf.placeholder(tf.int32, [None, 10, 1])
cell = tf.nn.rnn_cell.LSTMCell(1, dtype=tf.int32)
output, state = tf.nn.dynamic_rnn(cell=cell, inputs=X, dtype=tf.int32)
"><pre class="notranslate"><code class="notranslate">import tensorflow as tf

X = tf.placeholder(tf.int32, [None, 10, 1])
cell = tf.nn.rnn_cell.LSTMCell(1, dtype=tf.int32)
output, state = tf.nn.dynamic_rnn(cell=cell, inputs=X, dtype=tf.int32)

</code></pre></div>
<p dir="auto">The code below doesn't:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="
import tensorflow as tf

X = tf.placeholder(tf.float32, [None, 10, 1])
cell = tf.nn.rnn_cell.LSTMCell(1, dtype=tf.float32)
output, state = tf.nn.dynamic_rnn(cell=cell, inputs=X, dtype=tf.float32)
"><pre class="notranslate"><code class="notranslate">
import tensorflow as tf

X = tf.placeholder(tf.float32, [None, 10, 1])
cell = tf.nn.rnn_cell.LSTMCell(1, dtype=tf.float32)
output, state = tf.nn.dynamic_rnn(cell=cell, inputs=X, dtype=tf.float32)

</code></pre></div>
<p dir="auto">Note the change in dtype.</p>
<h2 dir="auto"><strong>Other info / logs</strong><br>
TRACEBACK:</h2>
<p dir="auto">TypeError                                 Traceback (most recent call last)<br>
 in ()<br>
2 X = tf.placeholder(tf.int32, [None, 10, 1])<br>
3 cell = tf.nn.rnn_cell.LSTMCell(1, dtype=tf.int32)<br>
----&gt; 4 output, state = tf.nn.dynamic_rnn(cell=cell, inputs=X, dtype=tf.int32)#, initial_state=state)<br>
5</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py in dynamic_rnn(cell, inputs, sequence_length, initial_state, dtype, parallel_iterations, swap_memory, time_major, scope)<br>
662         swap_memory=swap_memory,<br>
663         sequence_length=sequence_length,<br>
--&gt; 664         dtype=dtype)<br>
665<br>
666     # Outputs of _dynamic_rnn_loop are always shaped [time, batch, depth].</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py in _dynamic_rnn_loop(cell, inputs, initial_state, parallel_iterations, swap_memory, sequence_length, dtype)<br>
870       parallel_iterations=parallel_iterations,<br>
871       maximum_iterations=time_steps,<br>
--&gt; 872       swap_memory=swap_memory)<br>
873<br>
874   # Unpack final output if not using output tuples.</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py in while_loop(cond, body, loop_vars, shape_invariants, parallel_iterations, back_prop, swap_memory, name, maximum_iterations, return_same_structure)<br>
3289       ops.add_to_collection(ops.GraphKeys.WHILE_CONTEXT, loop_context)<br>
3290     result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants,<br>
-&gt; 3291                                     return_same_structure)<br>
3292     if maximum_iterations is not None:<br>
3293       return result[1]</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py in BuildLoop(self, pred, body, loop_vars, shape_invariants, return_same_structure)<br>
3002       with ops.get_default_graph()._mutation_lock():  # pylint: disable=protected-access<br>
3003         original_body_result, exit_vars = self._BuildLoop(<br>
-&gt; 3004             pred, body, original_loop_vars, loop_vars, shape_invariants)<br>
3005     finally:<br>
3006       self.Exit()</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py in _BuildLoop(self, pred, body, original_loop_vars, loop_vars, shape_invariants)<br>
2937         flat_sequence=vars_for_body_with_tensor_arrays)<br>
2938     pre_summaries = ops.get_collection(ops.GraphKeys._SUMMARY_COLLECTION)  # pylint: disable=protected-access<br>
-&gt; 2939     body_result = body(*packed_vars_for_body)<br>
2940     post_summaries = ops.get_collection(ops.GraphKeys._SUMMARY_COLLECTION)  # pylint: disable=protected-access<br>
2941     if not nest.is_sequence(body_result):</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/control_flow_ops.py in (i, lv)<br>
3258         cond = lambda i, lv: (  # pylint: disable=g-long-lambda<br>
3259             math_ops.logical_and(i &lt; maximum_iterations, orig_cond(*lv)))<br>
-&gt; 3260         body = lambda i, lv: (i + 1, orig_body(*lv))<br>
3261<br>
3262     if context.executing_eagerly():</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py in _time_step(time, output_ta_t, state)<br>
838           skip_conditionals=True)<br>
839     else:<br>
--&gt; 840       (output, new_state) = call_cell()<br>
841<br>
842     # Keras cells always wrap state as list, even if it's a single tensor.</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py in ()<br>
824     if is_keras_rnn_cell and not nest.is_sequence(state):<br>
825       state = [state]<br>
--&gt; 826     call_cell = lambda: cell(input_t, state)<br>
827<br>
828     if sequence_length is not None:</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py in <strong>call</strong>(self, inputs, state, scope, *args, **kwargs)<br>
368     # method.  See the class docstring for more details.<br>
369     return base_layer.Layer.<strong>call</strong>(self, inputs, state, scope=scope,<br>
--&gt; 370                                      *args, **kwargs)<br>
371<br>
372</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/layers/base.py in <strong>call</strong>(self, inputs, *args, **kwargs)<br>
372<br>
373       # Actually call layer<br>
--&gt; 374       outputs = super(Layer, self).<strong>call</strong>(inputs, *args, **kwargs)<br>
375<br>
376     if not context.executing_eagerly():</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py in <strong>call</strong>(self, inputs, *args, **kwargs)<br>
755       if not in_deferred_mode:<br>
756         self._in_call = True<br>
--&gt; 757         outputs = self.call(inputs, *args, **kwargs)<br>
758         self._in_call = False<br>
759         if outputs is None:</p>
<p dir="auto">~/jonassucks3/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py in call(self, inputs, state)<br>
1003            sigmoid(i + self._w_i_diag * c_prev) * self._activation(j))<br>
1004     else:<br>
-&gt; 1005       c = (sigmoid(f + self._forget_bias) * c_prev + sigmoid(i) *<br>
1006            self._activation(j))<br>
1007</p>
<p dir="auto">TypeError: unsupported operand type(s) for +: 'Tensor' and 'float'</p>
<p dir="auto">This code:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import tensorflow as tf

x = tf.constant([
		[[0,0],[5,0],[1,0],[1,0],[2,3],[4,0]],
		[[0,0],[0,0],[1,3],[2,0],[0,0],[0,0]]
	], dtype=tf.int32) #changing this to tf.float32 solves the problem


cell = tf.nn.rnn_cell.LSTMCell(num_units=15) 
initial_state = cell.zero_state(tf.shape(x)[0], dtype=tf.float32)
outputs, state = tf.nn.dynamic_rnn(cell, x, initial_state=initial_state, dtype=tf.float32)

init_op = tf.group(tf.global_variables_initializer(),
		tf.local_variables_initializer())

with tf.Session() as sess:
	sess.run(init_op)
	print(sess.run([outputs, state]))"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">tensorflow</span> <span class="pl-k">as</span> <span class="pl-s1">tf</span>

<span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">constant</span>([
		[[<span class="pl-c1">0</span>,<span class="pl-c1">0</span>],[<span class="pl-c1">5</span>,<span class="pl-c1">0</span>],[<span class="pl-c1">1</span>,<span class="pl-c1">0</span>],[<span class="pl-c1">1</span>,<span class="pl-c1">0</span>],[<span class="pl-c1">2</span>,<span class="pl-c1">3</span>],[<span class="pl-c1">4</span>,<span class="pl-c1">0</span>]],
		[[<span class="pl-c1">0</span>,<span class="pl-c1">0</span>],[<span class="pl-c1">0</span>,<span class="pl-c1">0</span>],[<span class="pl-c1">1</span>,<span class="pl-c1">3</span>],[<span class="pl-c1">2</span>,<span class="pl-c1">0</span>],[<span class="pl-c1">0</span>,<span class="pl-c1">0</span>],[<span class="pl-c1">0</span>,<span class="pl-c1">0</span>]]
	], <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">tf</span>.<span class="pl-s1">int32</span>) <span class="pl-c">#changing this to tf.float32 solves the problem</span>


<span class="pl-s1">cell</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-s1">nn</span>.<span class="pl-s1">rnn_cell</span>.<span class="pl-v">LSTMCell</span>(<span class="pl-s1">num_units</span><span class="pl-c1">=</span><span class="pl-c1">15</span>) 
<span class="pl-s1">initial_state</span> <span class="pl-c1">=</span> <span class="pl-s1">cell</span>.<span class="pl-en">zero_state</span>(<span class="pl-s1">tf</span>.<span class="pl-en">shape</span>(<span class="pl-s1">x</span>)[<span class="pl-c1">0</span>], <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">tf</span>.<span class="pl-s1">float32</span>)
<span class="pl-s1">outputs</span>, <span class="pl-s1">state</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-s1">nn</span>.<span class="pl-en">dynamic_rnn</span>(<span class="pl-s1">cell</span>, <span class="pl-s1">x</span>, <span class="pl-s1">initial_state</span><span class="pl-c1">=</span><span class="pl-s1">initial_state</span>, <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">tf</span>.<span class="pl-s1">float32</span>)

<span class="pl-s1">init_op</span> <span class="pl-c1">=</span> <span class="pl-s1">tf</span>.<span class="pl-en">group</span>(<span class="pl-s1">tf</span>.<span class="pl-en">global_variables_initializer</span>(),
		<span class="pl-s1">tf</span>.<span class="pl-en">local_variables_initializer</span>())

<span class="pl-k">with</span> <span class="pl-s1">tf</span>.<span class="pl-v">Session</span>() <span class="pl-k">as</span> <span class="pl-s1">sess</span>:
	<span class="pl-s1">sess</span>.<span class="pl-en">run</span>(<span class="pl-s1">init_op</span>)
	<span class="pl-en">print</span>(<span class="pl-s1">sess</span>.<span class="pl-en">run</span>([<span class="pl-s1">outputs</span>, <span class="pl-s1">state</span>]))</pre></div>
<p dir="auto">Does not work, because the inputs to the LSTM are integers and they need to be float. However, in version 1.4.0 I get this error:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="ValueError: Initializer for variable rnn/lstm_cell/kernel/ is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer."><pre class="notranslate"><code class="notranslate">ValueError: Initializer for variable rnn/lstm_cell/kernel/ is from inside a control-flow construct, such as a loop or conditional. When creating a variable inside a loop or conditional, use a lambda as the initializer.
</code></pre></div>
<p dir="auto">Which has nothing to do with what is wrong with the code. Version 1.2.0 however, generates this error which correctly refers to the problem:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="TypeError: Tensors in list passed to 'values' of 'ConcatV2' Op have types [int32, float32] that don't all match."><pre class="notranslate"><code class="notranslate">TypeError: Tensors in list passed to 'values' of 'ConcatV2' Op have types [int32, float32] that don't all match.
</code></pre></div>