What are the differences between type of layer and its activation function in PyTorch?
I am trying to write some simple neural network using pytorch. I am new to this library. I faced with two ways of implementing the same idea: a layer with some fixed activation function (e.g. tanh).

The first way to implement it:

l1 = nn.Tanh(n_in, n_out)


The second way:

l2 = nn.Linear(n_in, n_out) # linear layer, that do nothing with its input except summation


but in forward propagation use:

import torch.nn.functional as F
x = F.tanh(l2(x)) # x - value that propagates from layer to layer


What are the differences between those mechanisms? Which one is better for which purposes?

An activation function is just a non-linear function and it doesn't have any parameters. So, your first approach doesn't make any sense!

However, you can use a sequential wrapper to combine a linear layer with tanh activation.

model = nn.Sequential(
    nn.Linear(n_in, n_out),
    nn.Tanh()
)
output = model(input)

