title: Keras custom loss with one of the features used and a condition
body: <p>I'm trying to make a custom function for the deviance in Keras. 
  Deviance is calculated as : 2 * (log(yTrue) - log(yPred)) </p>
  
  <p>The problem here is that my yTrue values are rare event count and therefore often equal to 0, resulting in a -inf error. </p>
  
  <p>The derivation of deviance for my specific case (poisson unscaled deviance) gives a solution to this : </p>
  
  <ul>
  <li>If yTrue = 0, then deviance is : 2 * D * yPred where D is a feature of my data. </li>
  <li>If yTrue !=0, then deviance is : 2 * D * (yTrue * ln(yTrue) - yTrue * ln(yPred) - yTrue + yPred</li>
  </ul>
  
  <p>There is two problems here i encounter : </p>
  
  <ul>
  <li>I need to choose the function according to the value of yPred</li>
  <li>I also need to pass D as argument to the loss function </li>
  </ul>
  
  <p>I made a first iteration of a loss function before derivating deviance, adding small values to yTrue when it is equal to 0 to prevent the -Inf. problems, but it gives wrong results for deviance so i have to change it. </p>
  
  <pre><code>def DevianceBis(y_true, y_pred):
  y_pred = KB.maximum(y_pred, 0.0 + KB.epsilon()) #make sure ypred is positive or ln(-x) = NAN
  return (KB.sqrt(KB.square( 2 * KB.log(y_true + KB.epsilon()) - KB.log(y_pred))))
  </code></pre>
  
  <p>I'd like to know how to pass the D values into the loss function and how to use an if statement in order to choose the correct expression to use. </p>
  
  <p>Thanks in advance</p>
  
  <p>EDIT : </p>
  
  <p>Tried this one but returns NaN </p>
  
  <pre><code>    def custom_loss(data, y_pred):
  
          y_true = data[:, 0]
          d = data[:, 1:]
          # condition
          mask = keras.backend.equal(y_true, 0) #i.e. y_true != 0
          mask = KB.cast(mask, KB.floatx())
          # returns 0 when y_true =0, 1 otherwise
          #calculate loss using d...
          loss_value = mask * (2 * d * y_pred) + (1-mask) * 2 * d * (y_true * KB.log(y_true) - y_true * KB.log(y_pred) - y_true + y_pred)
          return loss_value
  
  
      def baseline_model():
          # create model
          #building model
          model = keras.Sequential()
          model.add(Dense(5, input_dim = 26, activation = "relu"))
          #model.add(Dense(10, activation = "relu"))
          model.add(Dense(1, activation = "exponential"))
          model.compile(loss=custom_loss, optimizer='RMSProp')
          return model
  
  model = baseline_model()
  model.fit(data2, np.append(y2, d, axis = 1), epochs=1, shuffle=True, verbose=1)
  </code></pre>
  
  <p>EDIT 2 : </p>
  
  <pre><code>def custom_loss(data, y_pred):
  
      y_true = data[:, 0]
      d = data[:, 1:]
      # condition
      mask2 = keras.backend.not_equal(y_true, 0) #i.e. y_true != 0
      mask2 = KB.cast(mask2, KB.floatx())
      # returns 0 when y_true =0, 1 otherwise
      #calculate loss using d...
      loss_value = 2 * d * y_pred + mask2 * (2 * d * y_true * KB.log(y_true) + 2 * d * y_true * KB.log(y_pred) - 2 * d * y_true)
      return loss_value
  </code></pre>
  
  <p>EDIT 3 seems to be working without the logs (altough it isn't the result i am looking for) : </p>
  
  <pre><code>def custom_loss(data, y_pred):
  
      y_true = data[:, 0]
      d = data[:, 1]
      # condition
      mask2 = keras.backend.not_equal(y_true, 0) #i.e. y_true != 0
      mask2 = KB.cast(mask2, KB.floatx())
      # returns 0 when y_true =0, 1 otherwise
      #calculate loss using d...
      loss_value = 2 * d * y_pred #+ mask2 * (2 * d * y_true * KB.log(y_true) + 2 * d * y_true * KB.log(y_pred) - 2 * d * y_true)
      return loss_value
  
  
  def baseline_model():
      # create model
      #building model
      model = keras.Sequential()
      model.add(Dense(5, input_dim = 26, activation = "relu"))
      #model.add(Dense(10, activation = "relu"))
      model.add(Dense(1, activation = "exponential"))
      model.compile(loss=custom_loss, optimizer='RMSProp')
      return model
  
  model = baseline_model()
  model.fit(data2, np.append(y2, d, axis = 1), epochs=1, shuffle=True, verbose=1)
  </code></pre>
  
  <p>EDIT again : </p>
  
  <pre><code>def custom_loss3(data, y_pred):
  
      y_true = data[:, 0]
      d = data[:, 1]
      # condition
      loss_value = KB.switch(KB.greater(y_true, 0), 2 * d * y_pred, 2 * d * (y_true * KB.log(y_true + KB.epsilon()) - y_true * KB.log(y_pred + KB.epsilon()) - y_true + y_pred))
      return loss_value
  </code></pre>