title: Using Subtract layer in Keras
body: <p>I'm implementing in Keras the LSTM architecture described <a href="http://nlp.cs.rpi.edu/paper/multilingualmultitask.pdf" rel="nofollow noreferrer">here</a>. I think I am really close, though I still have a problem with the combination of the shared and language-specific layers. Here is the formula (approximately): <code>y = g * y^s + (1 - g) * y^u</code></p>
  
  <p>And here is the code I tried:</p>
  
  <pre><code>### Linear Layers ###
  univ_linear = Dense(50, activation=None, name='univ_linear')
  univ_linear_en = univ_linear(en_encoded)
  univ_linear_es = univ_linear(es_encoded)
  print(univ_linear_en)
  
  # Gate &gt;&gt; g
  gate_en = Dense(50, activation='sigmoid', name='gate_en')(en_encoded)
  gate_es = Dense(50, activation='sigmoid', name='gate_es')(es_encoded)
  print(gate_en)
  print(gate_es)
  
  # EN &gt;&gt; y^s
  spec_linear_en = Dense(50, activation=None, name='spec_linear_en') (en_encoded)
  print(spec_linear_en)
  
  # g * y^s
  gated_spec_linear_en = Multiply()([gate_en, spec_linear_en])
  print(gated_spec_linear_en)
  
  # ES &gt;&gt; y^s
  spec_linear_es = Dense(50, activation=None, name='spec_linear_es')(es_encoded)
  print(spec_linear_es)
  
  # g * y^s
  gated_spec_linear_es = Multiply()([gate_es, spec_linear_es])
  print(gated_spec_linear_es)
  
  # 1 - Gate &gt;&gt; (1 - g)
  only_ones_en = K.ones(gate_en.shape)
  univ_gate_en = Subtract()([only_ones_en, gate_en])
  print(univ_gate_en)
  
  only_ones_es = K.ones(gate_es.shape)
  univ_gate_es = Subtract()([only_ones_es, gate_es])
  print(univ_gate_es)
  
  # (1 - g) * y^u
  gated_univ_linear_en = Multiply()([univ_gate_en, univ_linear_en])
  print(gated_univ_linear_en)
  gated_univ_linear_es = Multiply()([univ_gate_es, univ_linear_es])
  print(gated_univ_linear_es)
  
  out_en = Add()([gated_spec_linear_en, gated_univ_linear_en])
  print(out_en)
  
  out_es = Add()([gated_spec_linear_es, gated_univ_linear_es])
  print(out_es)
  </code></pre>
  
  <p>When I compile my model, I got this error:</p>
  
  <pre><code>AttributeError: 'NoneType' object has no attribute '_inbound_nodes'
  </code></pre>
  
  <p>Though, my model compiles without error when I replace <code>(1 - g) * y^u</code> by <code>g * y^u</code>:</p>
  
  <pre><code># (1 - g) * y^u
  gated_univ_linear_en = Multiply()([gate_en, univ_linear_en])
  print(gated_univ_linear_en)
  gated_univ_linear_es = Multiply()([gate_es, univ_linear_es])
  print(gated_univ_linear_es)
  </code></pre>
  
  <p>Consequently, I think the problem comes from the code under the comment <code># 1 - Gate &gt;&gt; (1 - g)</code>, and more precisely from the subtraction <code>(1 - g)</code>. </p>
  
  <p>Does anyone have any clue about what exactly the problem is and how I can solve it?</p>