title: What's wrong with the last two layers of this U-net?
body: <p>I'm building a U-net for a two-class image segmentation problem using Keras. I'm loosely following the example given here: <a href="https://github.com/zhixuhao/unet" rel="nofollow noreferrer">https://github.com/zhixuhao/unet</a> .  Following that example, the last few layers of the decoder (including the output layer) look like: </p>
  
  <pre><code>    up9 = UpSampling2D(size = (2,2))(conv8)
      merge9 = concatenate([conv1,up9], axis = 3)
      conv9 = Conv2D(16, 3, activation = 'relu', padding = 'same')(merge9)
      conv9 = Conv2D(16, 3, activation = 'relu', padding = 'same')(conv9)
  
      conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same')(conv9)
      conv10 = Conv2D(1, 1, activation = 'sigmoid',padding='same')(conv9)
  </code></pre>
  
  <p>The key problem that I'm finding is that when I remove this line:</p>
  
  <pre><code>conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same')(conv9)
  </code></pre>
  
  <p>then the U-net fails to train at all (using binary crossentropy), and I can't figure out why this layer is necessary given that I have seen almost identical U-nets elsewhere that skip this layer entirely.</p>
  
  <p>Can anyone help shed some light here on why this layer is necessary? If I were to make a WAG, I'd say it has something to do with the fact that we want to go down to 2 filters to represent each class? Does it have something to do with the expected output shape (128x128x1) in this case?</p>