Why doesn't nn.Sequential contain a softmax output layer in the example?
The example from PyTorch's official tutorial has the following ConvNet. My understanding is that the output layer uses a softmax to estimate the digit an image corresponds to. Why doesnt the code have a softmax layer or fully connected layer? 

model = nn.Sequential(
    nn.Conv2d(1, 16, kernel_size=3, stride=2, padding=1),
    nn.ReLU(),
    nn.Conv2d(16, 16, kernel_size=3, stride=2, padding=1),
    nn.ReLU(),
    nn.Conv2d(16, 10, kernel_size=3, stride=2, padding=1),
    nn.ReLU(),
    nn.AdaptiveAvgPool2d(1),
    Lambda(lambda x: x.view(x.size(0), -1)),
)

opt = optim.SGD(model.parameters(), lr=lr, momentum=0.9)


This is a very good question! The reason why no fully-connected layer is used is because of a technique called Global Average Pooling, implemented via nn.AdaptiveAvgPool2d(1). The benefits of this operation over fc layers were introduced in this paper, including reducing the number of model parameters while preserving performance, acting as a regulariser, and modelling deep localisation information. GAP can be used in place of fc, as well as before a subsequent fc layer.

As for why there is no softmax layer, I think that this is because they use the CrossEntropyLoss loss function in the backend. This function takes in raw logits and combines nn.LogSoftmax() and nn.NLLLoss() in one computation. So there is no need to perform an additional softmax function before loss evaluation.
