title: Changing thresholds in the Sigmoid Activation in Neural Networks
body: <p>Hi I am new to machine learning and I have a query about changing thresholds for sigmoid function.</p>
  
  <p>I know Sigmoid function's value is in the range [0;1], 0.5 is taken as a threshold, if h(theta) &lt; 0.5 we assume that it's value is 0, if h(theta) >= 0.5 then it's 1.</p>
  
  <p>Thresholds are used only on the output layer of the network and it's only when classifying. So, if you're trying to classify between 3 classes can you give different thresholds for each class (0.2,0.4,0.4 - for each class)? Or can you specify a different threshold overall, like 0.8? I am unsure how to define this in the code below. Any guidance is appreciated. </p>
  
  <pre><code># Hyper Parameters
  input_size = 14
  hidden_size = 40
  hidden_size2 = 30
  num_classes = 3
  num_epochs = 600
  batch_size = 34
  learning_rate = 0.01
  
  
  class Net(torch.nn.Module):
      def __init__(self, n_input, n_hidden, n_hidden2, n_output):
          super(Net, self).__init__()
          # define linear hidden layer output
          self.hidden = torch.nn.Linear(n_input, n_hidden)
          self.hidden2 = torch.nn.Linear(n_hidden, n_hidden2)
          # define linear output layer output
          self.out = torch.nn.Linear(n_hidden, n_output)
  
      def forward(self, x):
          """
              In the forward function we define the process of performing
              forward pass, that is to accept a Variable of input
              data, x, and return a Variable of output data, y_pred.
          """
          # get hidden layer input
          h_input1 = self.hidden(x)
          # define activation function for hidden layer
          h_output1 = torch.sigmoid(h_input1)
  
          # get hidden layer input
          h_input2 = self.hidden2(h_output1)
          # define activation function for hidden layer
          h_output2 = torch.sigmoid(h_input2)
  
          # get output layer output
          out = self.out(h_output2)
  
          return out
  
  
  net = Net(input_size, hidden_size, hidden_size, num_classes)
  
  criterion = nn.CrossEntropyLoss()
  optimizer = torch.optim.Adam(net.parameters(), lr=learning_rate)
  
  all_losses = []
  
  for epoch in range(num_epochs):
      total = 0
      correct = 0
      total_loss = 0
      for step, (batch_x, batch_y) in enumerate(train_loader):
          X = batch_x
          Y = batch_y.long()
  
      # Forward + Backward + Optimize
      optimizer.zero_grad()  # zero the gradient buffer
      outputs = net(X)
      loss = criterion(outputs, Y)
      all_losses.append(loss.item())
      loss.backward()
      optimizer.step()
  
      if epoch % 50 == 0:
          _, predicted = torch.max(outputs, 1)
          # calculate and print accuracy
          total = total + predicted.size(0)
          correct = correct + sum(predicted.data.numpy() == Y.data.numpy())
          total_loss = total_loss + loss
      if epoch % 50 == 0:
          print(
              "Epoch [%d/%d], Loss: %.4f, Accuracy: %.2f %%"
              % (epoch + 1, num_epochs, total_loss, 100 * correct / total)
          )
  
  train_input = train_data.iloc[:, :input_size]
  train_target = train_data.iloc[:, input_size]
  
  inputs = torch.Tensor(train_input.values).float()
  targets = torch.Tensor(train_target.values - 1).long()
  
  outputs = net(inputs)
  _, predicted = torch.max(outputs, 1)
  </code></pre>