<p dir="auto"><strong>System information</strong></p>
<ul dir="auto">
<li>Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes</li>
<li>OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10</li>
<li>Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A</li>
<li>TensorFlow installed from (source or binary): Binary, pip install</li>
<li>TensorFlow version (use command below): tensorflow-gpu==2.0.0-beta1</li>
<li>Python version: 3.6</li>
<li>Bazel version (if compiling from source): N/A</li>
<li>GCC/Compiler version (if compiling from source): N/A</li>
<li>CUDA/cuDNN version: N/A</li>
<li>GPU model and memory: N/A</li>
</ul>
<p dir="auto"><strong>Describe the current behavior</strong><br>
Error arises during Concatenate when I run the following code:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Conv2D, Concatenate

inputs = keras.Input(shape=(256,256,3))
x  = Conv2D(16,3, padding='same',activation='relu')(inputs)
x_list = [x]
for i in range(3):
    x = Conv2D(16,3, padding='same',activation='relu')(x)
    x_list.append(x)
    x = Concatenate(3)(x_list)

model = keras.Model(inputs=inputs, outputs=x)
model.summary()"><pre class="notranslate"><code class="notranslate">import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Conv2D, Concatenate

inputs = keras.Input(shape=(256,256,3))
x  = Conv2D(16,3, padding='same',activation='relu')(inputs)
x_list = [x]
for i in range(3):
    x = Conv2D(16,3, padding='same',activation='relu')(x)
    x_list.append(x)
    x = Concatenate(3)(x_list)

model = keras.Model(inputs=inputs, outputs=x)
model.summary()
</code></pre></div>
<p dir="auto"><code class="notranslate">ValueError: Graph disconnected: cannot obtain value for tensor Tensor("conv2d_31/Identity:0", shape=(None, 256, 256, 16), dtype=float32) at layer "concatenate_8". The following previous layers were accessed without issue: ['input_9', 'conv2d_29', 'conv2d_30']</code></p>
<p dir="auto">This issue does not occur in a Tensorflow 1.X environment, only TF 2.0</p>
<p dir="auto"><strong>Describe the expected behavior</strong><br>
Now the Concatenate function works properly when using a sequential model. That is, if I swap in "for i in range(1):" rather than "for i in range(3):" above, the code executes cleanly. However, the non-sequential repeated Concatenation in the loop leaves the  a Graph disconnected error.</p>
<p dir="auto">Furthermore, the error is also eliminated when using tf.concat, so the following code also executes cleanly.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Conv2D, Concatenate

inputs = keras.Input(shape=(256,256,3))
x  = Conv2D(16,3, padding='same',activation='relu')(inputs)
x_list = [x]
for i in range(3):
    x = Conv2D(16,3, padding='same',activation='relu')(x)
    x_list.append(x)
    x = tf.concat(x_list, 3)

model = keras.Model(inputs=inputs, outputs=x)
model.summary()"><pre class="notranslate"><code class="notranslate">import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Conv2D, Concatenate

inputs = keras.Input(shape=(256,256,3))
x  = Conv2D(16,3, padding='same',activation='relu')(inputs)
x_list = [x]
for i in range(3):
    x = Conv2D(16,3, padding='same',activation='relu')(x)
    x_list.append(x)
    x = tf.concat(x_list, 3)

model = keras.Model(inputs=inputs, outputs=x)
model.summary()
</code></pre></div>
<p dir="auto">Therefore, I do have a working alternative, but there does appear to be an issue with the keras Concatenate function</p>
<p dir="auto"><code class="notranslate">tf.keras.layers.Concatenate</code> used to operate in a very straightforward way with the Keras functional API for building DenseNet-esque feedforward networks.</p>
<p dir="auto">The code below works in TF 1.13 but fails in 1.14, 2.0.0a0 and beyond.</p>
<p dir="auto">I also tested replacing the <code class="notranslate">Concatenate</code> layer with its functional alternative, <code class="notranslate">concatenate</code>, and produced the same error.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from tensorflow.keras.layers import Dense, Input, Concatenate
from tensorflow.keras.models import Model

inputs = Input(shape=(10,))

all_layers = []

x1 = Dense(512)(inputs)
all_layers.append(x1)

# all layers: [x1]
x2 = Dense(256, activation='relu')(x1)
all_layers.append(x2)

# all layers: [x1, x2]
conc = Concatenate()(all_layers)
x3 = Dense(128, activation='relu')(conc)
all_layers.append(x3)

# all layers: [x1, x2, x3]
conc = Concatenate()(all_layers)
prediction = Dense(1)(conc)
model = Model(inputs=inputs, outputs=prediction)"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">tensorflow</span>.<span class="pl-s1">keras</span>.<span class="pl-s1">layers</span> <span class="pl-k">import</span> <span class="pl-v">Dense</span>, <span class="pl-v">Input</span>, <span class="pl-v">Concatenate</span>
<span class="pl-k">from</span> <span class="pl-s1">tensorflow</span>.<span class="pl-s1">keras</span>.<span class="pl-s1">models</span> <span class="pl-k">import</span> <span class="pl-v">Model</span>

<span class="pl-s1">inputs</span> <span class="pl-c1">=</span> <span class="pl-v">Input</span>(<span class="pl-s1">shape</span><span class="pl-c1">=</span>(<span class="pl-c1">10</span>,))

<span class="pl-s1">all_layers</span> <span class="pl-c1">=</span> []

<span class="pl-s1">x1</span> <span class="pl-c1">=</span> <span class="pl-v">Dense</span>(<span class="pl-c1">512</span>)(<span class="pl-s1">inputs</span>)
<span class="pl-s1">all_layers</span>.<span class="pl-en">append</span>(<span class="pl-s1">x1</span>)

<span class="pl-c"># all layers: [x1]</span>
<span class="pl-s1">x2</span> <span class="pl-c1">=</span> <span class="pl-v">Dense</span>(<span class="pl-c1">256</span>, <span class="pl-s1">activation</span><span class="pl-c1">=</span><span class="pl-s">'relu'</span>)(<span class="pl-s1">x1</span>)
<span class="pl-s1">all_layers</span>.<span class="pl-en">append</span>(<span class="pl-s1">x2</span>)

<span class="pl-c"># all layers: [x1, x2]</span>
<span class="pl-s1">conc</span> <span class="pl-c1">=</span> <span class="pl-v">Concatenate</span>()(<span class="pl-s1">all_layers</span>)
<span class="pl-s1">x3</span> <span class="pl-c1">=</span> <span class="pl-v">Dense</span>(<span class="pl-c1">128</span>, <span class="pl-s1">activation</span><span class="pl-c1">=</span><span class="pl-s">'relu'</span>)(<span class="pl-s1">conc</span>)
<span class="pl-s1">all_layers</span>.<span class="pl-en">append</span>(<span class="pl-s1">x3</span>)

<span class="pl-c"># all layers: [x1, x2, x3]</span>
<span class="pl-s1">conc</span> <span class="pl-c1">=</span> <span class="pl-v">Concatenate</span>()(<span class="pl-s1">all_layers</span>)
<span class="pl-s1">prediction</span> <span class="pl-c1">=</span> <span class="pl-v">Dense</span>(<span class="pl-c1">1</span>)(<span class="pl-s1">conc</span>)
<span class="pl-s1">model</span> <span class="pl-c1">=</span> <span class="pl-v">Model</span>(<span class="pl-s1">inputs</span><span class="pl-c1">=</span><span class="pl-s1">inputs</span>, <span class="pl-s1">outputs</span><span class="pl-c1">=</span><span class="pl-s1">prediction</span>)</pre></div>
<p dir="auto">The error output is</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="ValueError: Graph disconnected: cannot obtain value for tensor Tensor(&quot;dense_2/Identity:0&quot;, shape=(None, 128), dtype=float32) at layer &quot;concatenate&quot;. The following previous layers were accessed without issue: ['input_1', 'dense', 'dense_1']"><pre class="notranslate"><code class="notranslate">ValueError: Graph disconnected: cannot obtain value for tensor Tensor("dense_2/Identity:0", shape=(None, 128), dtype=float32) at layer "concatenate". The following previous layers were accessed without issue: ['input_1', 'dense', 'dense_1']
</code></pre></div>
<p dir="auto">Visualization of model<br>
<a target="_blank" rel="noopener noreferrer nofollow" href="https://camo.githubusercontent.com/8ef36ed034998d9dc422d3ee2a3130eaf8bbedbe76324be2058dc02dfaa4721b/68747470733a2f2f692e696d6775722e636f6d2f3137554e72536b2e706e67"><img src="https://camo.githubusercontent.com/8ef36ed034998d9dc422d3ee2a3130eaf8bbedbe76324be2058dc02dfaa4721b/68747470733a2f2f692e696d6775722e636f6d2f3137554e72536b2e706e67" alt="" data-canonical-src="https://i.imgur.com/17UNrSk.png" style="max-width: 100%;"></a></p>