title: Custom loss for deeplab
body: <p>I wanted to add custom loss for Deeplab v3 that would work for NOT one hot encoding labels but for saliency prediction. So instead of Deeplab loss implementation that you see below:</p>
  
  <pre><code>label = tf.to_int32(label &gt; 0.2)
  one_hot_labels = slim.one_hot_encoding(label, num_classes, on_value=1.0, off_value=0.0)
  tf.losses.softmax_cross_entropy(one_hot_labels, logits)
  </code></pre>
  
  <p>I used this implementation:</p>
  
  <pre><code>softmax = tf.log(tf.nn.softmax(logits))
  cross_entropy = -tf.reduce_sum(label*softmax, reduction_indices=[1])
  tf.losses.add_loss(tf.reduce_mean(cross_entropy))
  </code></pre>
  
  <p>Trained ~1000 epochs with 5 images and got this result: </p>
  
  <ul>
  <li>Input simplified image with padding -
  <a href="https://i.stack.imgur.com/07GsL.png" rel="noreferrer">https://i.stack.imgur.com/07GsL.png</a></li>
  <li>Ground truth labels - <a href="https://i.stack.imgur.com/ttEZi.png" rel="noreferrer">https://i.stack.imgur.com/ttEZi.png</a></li>
  <li>Custom loss result](<a href="https://i.stack.imgur.com/cNooX.png" rel="noreferrer">https://i.stack.imgur.com/cNooX.png</a></li>
  <li>Cross-entropy using one hot encoding result -
  <a href="https://i.stack.imgur.com/LEhl3.png" rel="noreferrer">https://i.stack.imgur.com/LEhl3.png</a></li>
  </ul>
  
  <p>Also, tried several learning rates, but it doesn't change the result for custom loss.</p>