Setting dimensions of layers in a convolutional neural network
Say I have 3x100x100 images in batches of 4 as input and I'm trying to make my first convolutional neural networks with pytorch. I'm really not sure if I'm getting convolutional neural networks right because when I train my input through the following arrangement I run into the error: 

Expected input batch_size (1) to match target batch_size (4).

The following is my forward nnet:

Then If I were to pass it through:

nn.Conv2d(3, 6, 5)


I would get 6 layers of maps each with dimensions (100-5+1). 

Then If I were to pass it through:

nn.MaxPool2d(2, 2)


I would get 6 layers of maps each with dimensions (96/2)

Then if I were to pass it through:

nn.Conv2d(6, 16, 5)


I would get 16 layers of maps each with dimensions (48-5+1)

Then If I were to pass it through:

self.fc1 = nn.Linear(44*44*16, 120)


I would get 120 neurons

Then If I were to pass it through:

self.fc2 = nn.Linear(120, 84)


I would get 84 neurons

Then If I were to pass it through: 

self.fc3 = nn.Linear(84, 3)


I would get 3 outputs which would be perfect because I have 3 classes of labels. But as I said before, this leads to an error which is really surprising because this makes a lot of sense to me.

Full neural network code:

import torch.nn as nn
import torch.nn.functional as F


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(44*44*16, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 3)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 *44*44)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x


net = Net()
net.to(device)


Your understanding is correct and very detailed. 

However, you have used two pooling layers (see relevant code below). So output after the second step will be 16 maps with 44/2=22 dimension.

x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))


To fix this either not pool or change the dimension of the fully-connected layer to 22*22*16. 

To fix by not pooling modify you forward function as below.

def forward(self, x):
    x = self.pool(F.relu(self.conv1(x)))
    x = F.relu(self.conv2(x))
    x = x.view(-1, 16 *44*44)
    x = F.relu(self.fc1(x))
    x = F.relu(self.fc2(x))
    x = self.fc3(x)
    return x


To fix by changing the dimension of the fully-connected layer, change the declaration of networks as below. 

def __init__(self):
    super(Net, self).__init__()
    self.conv1 = nn.Conv2d(3, 6, 5)
    self.pool = nn.MaxPool2d(2, 2)
    self.conv2 = nn.Conv2d(6, 16, 5)
    self.fc1 = nn.Linear(22*22*16, 120)
    self.fc2 = nn.Linear(120, 84)
    self.fc3 = nn.Linear(84, 10)

