Pytorch loss does't change in vgg 19 model
In pytorch I made a model vgg19 for classification tiny imagenet:

model = nn.Sequential(
    nn.BatchNorm2d(3),
    nn.Conv2d(in_channels=3, out_channels=64, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=64, out_channels=64, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.MaxPool2d((2,2)),
    nn.Conv2d(in_channels=64, out_channels=128, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=128, out_channels=128, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.MaxPool2d((2,2)),
    nn.Conv2d(in_channels=128, out_channels=256, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=256, out_channels=256, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=256, out_channels=256, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=256, out_channels=256, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.MaxPool2d((2,2)),
    nn.Conv2d(in_channels=256, out_channels=512, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=512, out_channels=512, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=512, out_channels=512, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=512, out_channels=512, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.MaxPool2d((2,2)),
    nn.Conv2d(in_channels=512, out_channels=512, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=512, out_channels=512, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=512, out_channels=512, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.Conv2d(in_channels=512, out_channels=512, kernel_size=(3,3), padding=1),
    nn.ReLU(),
    nn.MaxPool2d((2,2)),
    nn.Flatten(),
    nn.Linear(25088, 4096),
    nn.Linear(4096, 1000),
    nn.Linear(1000, 200),
    nn.Softmax(),
    nn.Dropout2d(),
)


In the learning process, the loss remains at approximately the same value (5.3 +- 0.01).

###############
iter = 4000 / 80000
loss = 5.295811176300049
###############
iter = 4800 / 80000
loss = 5.298299789428711
###############
iter = 5600 / 80000
loss = 5.309792995452881
###############
iter = 6400 / 80000
loss = 5.3179707527160645
###############
iter = 7200 / 80000
loss = 5.3179707527160645


I have already tried to increase and decrease lr, batch_size. But still no idea how to fix it.

Training code
(epochs = 10, loss = cross_entropy, optim = Adam(lr = 0.01), X_batch.shape = (batch_size, 3, 224, 224):

for epoch in range(num_epochs):
    i = 0
    for (X_batch, y_batch) in train_batch_gen:
        X_batch = Variable(torch.FloatTensor(X_batch)).cuda()
        y_batch = Variable(torch.LongTensor(y_batch)).cuda()
        logits = model.cuda().forward(X_batch)
        opt.zero_grad()
        loss = lossFunc(logits, y_batch)
        loss.backward()
        opt.step()

        train_loss.append(loss.data.cpu().numpy())
        if i % (batch_size*100) == 0:
            print("###############")
            print(f"iter = {i} / {80000}")
            print(f"loss = {np.mean(train_loss[-len(train_dataset) // batch_size :])}")
        i += batch_size


nn.CrossEntropyLoss applies log-softmax, but you also apply softmax in the model:

nn.Linear(1000, 200),
nn.Softmax(),
nn.Dropout2d(),


The output of your model must be the raw logits, without the nn.Softmax().

Additionally, dropout should not be used just before the output of the model, since that effectively wipes out some of the classes, making the loss punish something that would have been correct otherwise. Dropout should only be used between layers as a regularisation.
