title: Create custom 'non differentiable' activation function in keras
body: <p>Is it possible to create a custom activation function of the form:</p>
  
  <pre><code>def newactivation(x):
      if x &lt;= -1:
          return -1
      elif x &gt; -1 and x &lt;= 1
          return x
      else :
          return 1
  </code></pre>
  
  <p>So basically it would be a linearized version of tanh(x).</p>
  
  <p>Is it a problem during optimization, that the function has two non differentiable positions at -1 and 1?</p>
  
  <p>How could I implement this?</p>