Wrong value of standard deviation
Hello I am trying to evaluate the standard deviation and the mean of the dataset MNIST and I get a wrong value for the standard deviation. Here is my code :

import torch
from torchvision import datasets, transforms
import torch.nn.functional as F

loader = torch.utils.data.DataLoader(datasets.MNIST(
'../data', train=True, download=True, transform=transform1),
                     batch_size=32,
                     num_workers=0,
                     shuffle=False)

mean = 0.
std = 0.
for images, _ in loader:
    batch_samples = images.size(0) 
    images = images.view(batch_samples, images.size(1), -1)
    mean += images.mean(2).sum(0)
    std += images.std(2).sum(0)

mean /= len(loader.dataset)
std /= len(loader.dataset)

print("The mean is ", mean)
print("The standard deviation is ", std)


My problem is the following, I get as mean the value 0.1307 and for the standard deviation the value 0.3015 instead of 0.3081. I suppose I have an error in my code but I don't see where. 

Could you help me please ?

Thank you very much !

torch.std uses the batches mean as part of the computation so it's not the same as using torch.std on the entire dataset since that would use a different mean. We can use the following well known expression for variance to get the desired result

Var(X) = E[X**2] - E[X]**2

mean = 0.
mean_square = 0.
samples = 0
for images, _ in loader:
    batch_samples = images.size(0) 
    images = images.view(batch_samples, images.size(1), -1)
    mean += images.mean(2).sum(0)
    mean_square += (images**2).mean(2).sum(0)
    samples += images.size(2) * images.size(0)

mean /= len(loader.dataset)
mean_square /= len(loader.dataset)

# extra scale factor for unbias std estimate (it's effectively 1.0)
scale = samples / (samples - 1)
std = torch.sqrt((mean_square - mean**2) * scale)

print("The mean is ", mean)
print("The standard deviation is ", std)


Of course in the special case of the torchvision MNIST dataset you could just directly compute mean and standard deviation...

mean = torch.mean(loader.dataset.data.float() / 255.0)
std = torch.std(loader.dataset.data.float() / 255.0)

