What is the derivative of the Leaky ReLU activation function?
Derivative gives the rate of change in $y$ for a small change in $x$ or the slope of a function at point $x$.

In the above function,

y = x      for x &gt;= 0,     i.e. y/x = 1
y = x/20   for x &lt; 0,      i.e.  y/x = 1/20


The following function returns the derivative of leaky ReLU as explained

private double leaky_relu_derivative(double x)
{
    if (x &gt;= 0)
        return 1;
    else
        return 1.0 / 20;
}