What exactly is the "hyperbolic" tanh function used in the context of activation functions?
The hyperbolic trig functions follow the equation for a Rectangular hyperbola, which is something you should be familiar from analytical geometry.  The recentgular hyperbola is defined by $x^2 - y^2 = 1$ and if you let $x = cosh(t)$ and $y = sinh(t)$ and plug it into the rectangular hyperbola equation you can verify this fact quite easily.

From this idea you can derive the rest of the hyperbolic trig functions.  (Also they reason they are given trig names is because they resemble the properties trig functions have).