Is ReLU a non-linear activation function?
Short Answer: Yes

Visually:



if you see the image from wikipedia, it shown that ReLU (the blue line) is non-Linear (the line is not straight, it turns in 0). You can also check "visual" definition of linear function in wikipedia:


  "In calculus and related areas, a linear function is a function whose graph is a straight line"


Mathematically:

Linear function of one variable can be defined as: 

$ f(x) = ax + b $ 

If you plot that function in 2D, it will give you a straight line. Then, the form of linear function with multi variables:

$ f(x_1, x_2, ..., x_n) = a_1x_1 + a_2x_2 + ... + a_nx_n + b $

If you again plot that function in the correct dimension it also give you a straight line. And if you that function carefully, it similar with calculation that happen in a neuron. That's why neuron addition and multiplication is a linear function:

$ f(x_1, x_2, ..., x_n) = w_1x_1 + w_2x_2 + ... + w_nx_n + b $

Adding more layer of linear functions doesn't make the function become "complex" for example, if you have $f(x)$ like below and then you put another layer of linear function $g(x)$ on top of it:

$f(x) = ax + b$

$g(x) = cf(x) + d = cax + cb + d$

as the neural network is trained to find the value of $a,b,c,d$, we can group the constant from the formula above, and then rewrite to:

$h(x) = mx + n$

with $m=ca$ and $n=cb+d$. So without non-linear function the layer of neural network is useless, it only give you another "simple" linear function

ReLU formula is a $f(x)=max(0,x)$, it produces non-linearity as you can't write to linear function format. Using this function will give you "complexity" when you add more layer on top of it.