Activation function vs Squashing function This may seem like a very simple and obvious question, but I haven't actually been able to find a direct answer.
Today, in a video explaining deep neural networks, I came across the term Squashing function. This is a term that I have never heard or used. Our professor always used the term Activation function instead. Given the definitions I've been able to find, the two seem to be interchangeable terms.
Are they really synonymous or is there a difference?
Activation functions like sigmoid function, hyperbolic tangent function, etc. are also called squashing function because they squash the input into a small range like in sigmoid function output is in range of [-1,1]. But you cannot call ReLU as a squashing function because for a positive input value it returns the output as same.