title: Pytorch Lightning: High Loss Value
body: <p>I am trying to train 3 models with Pytroch Lightning. These would be 'eca_resnet50t', 'efficientnet_b3', 'vit_base_patch16_224'. I use the pretrained models from <a href="https://github.com/FrancescoSaverioZuppichini/glasses" rel="nofollow noreferrer">Glasses</a> for this. The code works, but I get strange values as Loss. I use CrossEntropyLoss and get initial values between 1000-200, although the Accuracy is quite high. I don't know what the reason is. I also tried to train the models only in Pytorch and get there &quot;normal&quot; values of initially 1.3 and after 5 epochs about values of 0.3.
  What may be the problem?</p>
  <p>Here is my LightningModule class:</p>
  <pre><code>class ClassifikationTask(pl.LightningModule):
  def __init__(self, model_name, num_classes=4, batch_size=64, lr_rate=0.01, pretrained=True, augment=&quot;Full&quot;):
      super().__init__()
      print(&quot;-----------NEW---------&quot;)
      self.num_classes = num_classes
      self.batch_size = batch_size
      self.model_name = model_name
      self.loss = nn.CrossEntropyLoss()
      self.lr = lr_rate
      self.save_hyperparameters()
      
      #getting the model
      if pretrained:
        print(&quot;PRETRAINED!&quot;)
        self.model = AutoModel.from_pretrained(self.model_name)
      else: 
        self.model = AutoModel.from_name(self.model_name)
  
      self.model.freeze()
  
      if model_name == 'efficientnet_b3':
        self.model.head = EfficientNetHead(in_features=self.model.encoder.widths[-1], n_classes=self.num_classes)
      elif model_name == 'eca_resnet50t':
        self.model.head = ResNetHead(in_features=self.model.encoder.widths[-1], n_classes=self.num_classes)
      else:
        self.model.head = ViTClassificationHead(n_classes=4)
  
  
      
  
      # no grad in the encoder
      for param in self.model.encoder.parameters():
          assert not param.requires_grad
      # grad in the head
      for param in self.model.head.parameters():
          assert param.requires_grad
  
  def forward(self, x):
      out = self.model.forward(x)
      return out
  
  def training_step(self, batch, batch_idx):
      loss, acc = self._shared_eval_step(batch, batch_idx)
      metrics = {&quot;train_acc&quot;: acc, &quot;train_loss&quot;: loss}
      self.log_dict(metrics, batch_size=self.batch_size)
      return loss
  
  def validation_step(self, batch, batch_idx):
      loss, acc = self._shared_eval_step(batch, batch_idx)
      metrics = {&quot;val_acc&quot;: acc, &quot;val_loss&quot;: loss}
      self.log_dict(metrics, batch_size=self.batch_size)
      return loss
  
  def test_step(self, batch, batch_idx):
      loss, acc = self._shared_eval_step(batch, batch_idx)
      metrics = {&quot;test_acc&quot;: acc, &quot;test_loss&quot;: loss}
      self.log_dict(metrics, batch_size=self.batch_size )
      return loss
  
  def _shared_eval_step(self, batch, batch_idx):
      x, y = batch
      y_hat = self(x)
      loss = self.loss(y_hat, y)
      acc = FM.accuracy(y_hat, y)
      return loss, acc
  
  def configure_optimizers(self):
      optimizer = torch.optim.Adam(self.model.parameters(), lr=self.lr)
      scheduler_dict = {
          &quot;scheduler&quot;: ReduceLROnPlateau(
              optimizer,
              factor=0.1,
              patience=3,
              verbose=True
          ),
          &quot;monitor&quot;: &quot;val_loss&quot;,
          'interval': 'epoch',
      }
      return {&quot;optimizer&quot;: optimizer, &quot;lr_scheduler&quot;: scheduler_dict}
  </code></pre>