Spaces:
Running
on
Zero
Running
on
Zero
File size: 483 Bytes
29d49a9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
"""
Script based on:
Wang, Xueliang, Honge Ren, and Achuan Wang.
"Smish: A Novel Activation Function for Deep Learning Methods.
" Electronics 11.4 (2022): 540.
"""
# import pytorch
import torch
@torch.jit.script
def smish(input):
"""
Applies the mish function element-wise:
mish(x) = x * tanh(softplus(x)) = x * tanh(ln(1 + exp(sigmoid(x))))
See additional documentation for mish class.
"""
return input * torch.tanh(torch.log(1 + torch.sigmoid(input)))
|