Torch.nn has a specific activation function?
My network has a output layer with Relu activation function, but I want the output is something like &quot;Relu+1&quot;, that is I want the output is all bigger than 1 and has the same shape of Relu function.
How should I change my torch.nn network?
My code is like:
self.actor = nn.Sequential(
                     nn.Linear(state_dim, 256),
                     nn.ReLU(),
                     nn.Linear(256, 256),
                     nn.ReLU(),
                     nn.Linear(256, action_dim),
                     nn.ReLU()
)


There are 2 ways I can think of.

Make self.actor an nn.Module object

class Actor(nn.Module):
    def __int__(self, state_dim, action_dim):
        super().__init__()
        self.linear1 = nn.Linear(state_dim, 256)
        self.relu = nn.ReLU()
        self.linear2 = nn.Linear(256, 256)
        self.linear3 = nn.Linear(256, action_dim)

    def forward(self, x):
        x = self.linear1(x)
        x = self.relu(x)
        x = self.linear2(x)
        x = self.relu(x)
        x = self.linear2(x)
        x = self.relu(x) + 1
        return x

class ......
    self.actor = Actor(state_dim, action_dim)


Make a Module class to do that and add it to the self.actor

class Add1(nn.Module):
    def forward(self, x):
        return x + 1

class ......
    self.actor = nn.Sequential(
                     nn.Linear(state_dim, 256),
                     nn.ReLU(),
                     nn.Linear(256, 256),
                     nn.ReLU(),
                     nn.Linear(256, action_dim),
                     nn.ReLU(),
                     Add1()
)

