Numerically stable hyperbolic tangent The hyperbolic tangent is commonly used as an activation function:
$$
tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}}
$$
Although, it is unclear how this function is implemented to be numerically stable in frameworks like pytorch or numpy. Since $e^x$ will explode for any $x \gtrapprox 1000$, is the function being approximated in some way? Or are there static checks for big input numbers?
Commonly you divide the domain up into different accuracy,stability regimes and apply a different approximation within each.
For $|x|&lt;1$ you can use a polynomial approximation, like Chebyshev. For $1&lt;|x|\leq E$, $E$ depending on implementation of the formula, you can use a related formula to the one you described.
For $|x|&gt;E$ you can use $\tanh(x) \approx sign(x).$
More details here from a documented implementation.