How to prevent inf while working with exponential
I'm trying to create  a function in a network with trainable parameters. In my function I have an exponential that for large tensor values goes to infinity. What would the best way to avoid this be? 

The function is as follows:

step1 = Pss-(k*Pvv)
step2 = step1*s
step3 = torch.exp(step2)
step4 = torch.log10(1+step3)
step5 = step4/s

#or equivalently
# train_curve = torch.log(1+torch.exp((Pss-k*Pvv)*s))/s


If it makes it easier to understand, the basic function is log10(1+e^(x-const)*10)/10. The exponential inside the log gets too big and goes to inf. 

I think I might have to normalize my tensor x, and this would mean normalizing the constants and the rest of the function also. Would someone have any thoughts on the best way to go about this?  

Thanks so much.

One solution is to just use a more stable computation. Notice that log(1 + exp(x)) is approximately equal to x when x is large enough. Intuitively this can be observed by noting that, for example, exp(50) is approximately 5.18e+21 for which adding 1 will have no effect when using 32-bit floating point arithmetic like PyTorch does. Further verification using an arbitrary precision calculator shows that the error in this approximation at 50 is far outside the maximum 32-bit floating point precision (which is about 7 decimal digits).
Using this information we can implement a simple piecewise function in PyTorch for which we use log1p(exp(x)) for values less than 50 and x for values greater than 50. Also note that this function is autograd compatible
def log1pexp(x):
    # more stable version of log(1 + exp(x))
    return torch.where(x &lt; 50, torch.log1p(torch.exp(x)), x)

This get's us most of the way to a solution since you actually want to evaluate torch.log10(1+torch.exp((Pss-k*Pvv)*s))/s
Now we can use our new log1pexp function to compute this expression without worrying about infinities
(log1pexp((Pss - k*Pvv)*s) / math.log(10)) / s

and mind the conversion from natural log to log base-10 by dividing by log(10).
