title: Why network with linear layers can't learn anything
body: <p>I am trying to build a neural network for binary classification, unfortunately it always predicts the value 0, even though one fifth of the training set data is 1. I have no idea why it is so. My dataset looks as this, so there are a couple of categorical variables and a couple of continuous, (target is the one we predict):</p>
  <p><a href="https://i.stack.imgur.com/Os1qL.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Os1qL.png" alt="enter image description here" /></a></p>
  <p>Here one can download the data:
  <a href="https://drive.google.com/drive/folders/1PsG2rRdbxyocyqvLSa7zSy_aVDMRJ2Ug?usp=sharing" rel="nofollow noreferrer">https://drive.google.com/drive/folders/1PsG2rRdbxyocyqvLSa7zSy_aVDMRJ2Ug?usp=sharing</a></p>
  <p>You can read it with</p>
  <pre><code>df = pd.read_csv(&quot;train.csv&quot;, index_col=0)
  </code></pre>
  <p>Now I am preparing the data for neural network:</p>
  <pre><code>x_train=df.drop(labels=['target'], axis=1).values
  y_train=df['target'].values
  X_train, X_val, y_train, y_val = train_test_split(
  x_train,
  y_train,
  test_size=0.2)
  
  LR = 0.001 
  EPOCH = 50
  BATCH_SIZE = 64
  
  torch_X_train = torch.tensor(X_train)
  torch_y_train = torch.tensor(y_train)
  
  torch_X_val = torch.tensor(X_val)
  torch_y_val = torch.tensor(y_val)
  
  train = torch.utils.data.TensorDataset(torch_X_train,torch_y_train)
  validate = torch.utils.data.TensorDataset(torch_X_val,torch_y_val)
  
  train_loader = torch.utils.data.DataLoader(train, batch_size = 
  BATCH_SIZE, shuffle = True)
  val_loader = torch.utils.data.DataLoader(validate, batch_size = 
  BATCH_SIZE, shuffle = False)
  </code></pre>
  <p>And define a simple network with just linear layers:</p>
  <pre><code>class NN(nn.Module):
  def __init__(self, input_size):
      super().__init__()
      self.to_class=nn.Sequential(
      nn.Linear(input_size,512),
      nn.ReLU(),
      nn.Linear(512,256),
      nn.ReLU(),
      nn.Linear(256,32),
      nn.ReLU(),
      nn.Linear(32,2)
      )
  def forward(self,inputs):
      pred= self.to_class(inputs)
      return F.softmax(pred, dim=1)
  </code></pre>
  <p>And lastly I train it</p>
  <pre><code>net=NN(7)
  
  
  loss_func = nn.CrossEntropyLoss()
  optimizer = optim.Adam(net.parameters(), lr=LR)
  
  train_loss = np.zeros(EPOCH)
  val_loss = np.zeros(EPOCH)
  acc_train = []
  acc_val=[]
  
  for epoch in range(EPOCH):
      correct = 0
      total = 0
      for data in train_loader:
          X, y = data
          optimizer.zero_grad()
          output=net(X.float())
          total += output.size(0)
          pred = output.argmax(dim=1, keepdim=True)
          correct += pred.eq(y.view_as(pred)).sum().item()
          loss = loss_func(output.squeeze(), y) 
          loss.backward()                     
          optimizer.step() 
          train_loss[epoch] += loss
      train_loss[epoch] /= len(train_loader)
      acc_train.append(correct/total*100)
  print('epoch %d:\t train_accuracy %.5f\ttrain loss: %.5f'%(epoch,acc_train[epoch], train_loss[epoch]))
  </code></pre>
  <p>But the train loss is not going anywhere and the predictions are always one class! Can someone explain this phenomenon and hint how I can improve it?</p>
  <p><a href="https://i.stack.imgur.com/IsnFz.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/IsnFz.png" alt="enter image description here" /></a></p>