title: PyTorch Neural Net not learning
body: <p>I am new to PyTorch and I'm trying to build a simple neural net for classification. The problem is the network doesn't learn at all. I tried various learning rate ranging from 0.3 to 1e-8 and I also tried training it for a longer duration. My data is small with only 120 training examples and the batch size is 16. Here is the code</p>
  <p>Define network</p>
  <pre><code>model = nn.Sequential(nn.Linear(4999, 1000),
                    nn.ReLU(),
                    nn.Linear(1000,200),
                    nn.ReLU(),
                    nn.Linear(200,1),
                    nn.Sigmoid())
  </code></pre>
  <p>Loss and optimizer</p>
  <pre><code>import torch.optim as optim
  
  optimizer = optim.SGD(model.parameters(), lr=0.01)
  criterion = nn.BCELoss(reduction=&quot;mean&quot;)
  </code></pre>
  <p>Training</p>
  <pre><code>num_epochs = 100
  for epoch in range(num_epochs):
    cumulative_loss = 0
    for i, data in enumerate(batch_gen(X_train, y_train, batch_size=16)):
      inputs, labels = data
      inputs = torch.from_numpy(inputs).float()
      labels = torch.from_numpy(labels).float()
  
      optimizer.zero_grad()
  
      outputs = model(inputs)
      loss = criterion(outputs, labels)
      loss.backward()
      optimizer.step()
      cumulative_loss += loss.item()
  
      if i%5 == 0 and i != 0:
        print(f&quot;epoch {epoch} batch {i} =&gt; Loss: {cumulative_loss/5}&quot;)
    
  
  
  print(&quot;Finished Training!!&quot;)
  </code></pre>
  <p>Any help is appreciated!</p>