PyTorch: accuracy of validation set greater than 100% during training
1 ) Problem

I observe an odd behaviour during training where my validation-accuracy is above 100% right from the start.

Epoch 0/3
----------
100%|██████████| 194/194 [00:50&lt;00:00,  3.82it/s]
train Loss: 1.8653 Acc: 0.4796
100%|██████████| 194/194 [00:32&lt;00:00,  5.99it/s]
val Loss: 1.7611 Acc: 1.2939

Epoch 1/3
----------
100%|██████████| 194/194 [00:42&lt;00:00,  4.61it/s]
train Loss: 0.8704 Acc: 0.7467
100%|██████████| 194/194 [00:31&lt;00:00,  6.11it/s]
val Loss: 1.0801 Acc: 1.4694


The output indicates that one epoch iterates over 194 batches, which does seem to be  correct for the training data (which has a length of 6186, batch_size is 32, hence 32*194 = 6208 and this is ≈6186) but does not match the size of the validation-data (length of 3447, batch_size = 32). 

Hence I would expect my validation-loop to generate 108 (3447 / 32 ≈ 108) batches insted of 194. 

I thought this behaviour is handled within my for loop at:

for dataset in tqdm(dataloaders[phase]):


But somehow I can't figure out what is wrong here. See point 3) below for my entire code. 

2 ) Question

If my assumption above is correct i.e. that this error stems from the for-loop within in my code then I would like to know the following: 

How do I need to adjust the for-loop during the validation phase to handle the number of batches that are being used for validation correctly?

3 ) Background:

Following two tutorials, one on how to do transfer-learning (https://discuss.pytorch.org/t/transfer-learning-using-vgg16/20653) and one on how to do data-loading (https://pytorch.org/tutorials/beginner/data_loading_tutorial.html) in pytorch, I am trying to customize the code such that I can perform transfer-learning on a new custom dataset which I want to provide via pandas dataframes.

As such, my training- and validation-data is provided via two dataframes (df_train &amp; df_val) which both contain two columns, one for the path and one for the target. E.g. like this:

    url                                 target
0   C:/Users/aaron/Desktop/pics/4ebd... 9
1   C:/Users/aaron/Desktop/pics/7153... 3
2   C:/Users/aaron/Desktop/pics/3ee6... 3
3   C:/Users/aaron/Desktop/pics/4652... 16
4   C:/Users/aaron/Desktop/pics/28ce... 15
...


And their respective length:

print(len(df_train))
print(len(df_val))
&gt;&gt; 6186
&gt;&gt; 3447


My pipeline looks like this:

class CustomDataset(Dataset):
    def __init__(self, df, transform=None):

        self.dataframe = df_train
        self.transform = transform

    def __len__(self):
        return len(self.dataframe)

    def __getitem__(self, idx):
        img_name = self.dataframe.iloc[idx, 0]
        img = Image.open(img_name)
        img_normalized = self.transform(img)

        landmarks = self.dataframe.iloc[idx, 1]
        sample = {'data': img_normalized, 'label': int(landmarks)}

        return sample

train_dataset = CustomDataset(df_train,transform=transforms.Compose([
                                                transforms.Resize(224),
                                               transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]))

val_dataset = CustomDataset(df_val,transform=transforms.Compose([
                                                transforms.Resize(224),
                                               transforms.ToTensor(),transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])]))

train_loader = torch.utils.data.DataLoader(train_dataset,batch_size=32,shuffle=True, num_workers=0)
val_loader = torch.utils.data.DataLoader(val_dataset,batch_size=32,shuffle=True, num_workers=0)

dataloaders = {'train': train_loader, 'val': val_loader}
dataset_sizes = {'train': len(df_train) ,'val': len(df_val)}


################### Training

from tqdm import tqdm

def train_model(model, criterion, optimizer, scheduler, num_epochs=25):
    since = time.time()

    best_model_wts = copy.deepcopy(model.state_dict())
    best_acc = 0.0

    for epoch in range(num_epochs):
        print('Epoch {}/{}'.format(epoch, num_epochs - 1))
        print('-' * 10)

        # Each epoch has a training and validation phase
        for phase in ['train', 'val']:
            if phase == 'train':
                scheduler.step()
                model.train()  # Set model to training mode
            else:
                model.eval()   # Set model to evaluate mode

            running_loss = 0.0
            running_corrects = 0

            # Iterate over data.
            for dataset in tqdm(dataloaders[phase]):

                inputs, labels = dataset["data"], dataset["label"]
                #print(inputs.type())
                inputs = inputs.to(device, dtype=torch.float)
                labels = labels.to(device,dtype=torch.long)

                # zero the parameter gradients
                optimizer.zero_grad()

                # forward
                # track history if only in train
                with torch.set_grad_enabled(phase == 'train'):
                    outputs = model(inputs)
                    _, preds = torch.max(outputs, 1)
                    loss = criterion(outputs, labels)

                    # backward + optimize only if in training phase
                    if phase == 'train':
                        loss.backward()
                        optimizer.step()

                # statistics
                running_loss += loss.item() * inputs.size(0)
                running_corrects += torch.sum(preds == labels.data)

            epoch_loss = running_loss / dataset_sizes[phase]
            epoch_acc = running_corrects.double() / dataset_sizes[phase]

            print('{} Loss: {:.4f} Acc: {:.4f}'.format(
                phase, epoch_loss, epoch_acc))

            # deep copy the model
            if phase == 'val' and epoch_acc &gt; best_acc:
                best_acc = epoch_acc
                best_model_wts = copy.deepcopy(model.state_dict())

        print()

    time_elapsed = time.time() - since
    print('Training complete in {:.0f}m {:.0f}s'.format(
        time_elapsed // 60, time_elapsed % 60))
    print('Best val Acc: {:4f}'.format(best_acc))

    # load best model weights
    model.load_state_dict(best_model_wts)
    return model


device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

model_ft = models.resnet18(pretrained=True)
num_ftrs = model_ft.fc.in_features
model_ft.fc = nn.Linear(num_ftrs, len(le.classes_))

model_ft = model_ft.to(device)

criterion = nn.CrossEntropyLoss()

# Observe that all parameters are being optimized
optimizer_ft = optim.SGD(model_ft.parameters(), lr=0.001, momentum=0.9)

# Decay LR by a factor of 0.1 every 7 epochs
exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=7, gamma=0.1)

model_ft = train_model(model_ft, criterion, optimizer_ft, exp_lr_scheduler,
                       num_epochs=4)


Your problem appears to be here: 

class CustomDataset(Dataset):
    def __init__(self, df, transform=None):
&gt;&gt;&gt;&gt;&gt;        self.dataframe = df_train


This should be 

             self.dataframe = df


In your case, you are inadvertently setting both the train and val CustomDataset to df_train ...
