Does the reduction of the dimensions over multiple layers allow more details to be stored within the final representation?
From : https://debuggercafe.com/implementing-deep-autoencoder-in-pytorch/
the following autoencoder is defined

class Autoencoder(nn.Module):
    def __init__(self):
        super(Autoencoder, self).__init__()

        # encoder
        self.enc1 = nn.Linear(in_features=784, out_features=256)
        self.enc2 = nn.Linear(in_features=256, out_features=128)
        self.enc3 = nn.Linear(in_features=128, out_features=64)
        self.enc4 = nn.Linear(in_features=64, out_features=32)
        self.enc5 = nn.Linear(in_features=32, out_features=16)

        # decoder 
        self.dec1 = nn.Linear(in_features=16, out_features=32)
        self.dec2 = nn.Linear(in_features=32, out_features=64)
        self.dec3 = nn.Linear(in_features=64, out_features=128)
        self.dec4 = nn.Linear(in_features=128, out_features=256)
        self.dec5 = nn.Linear(in_features=256, out_features=784)

    def forward(self, x):
        x = F.relu(self.enc1(x))
        x = F.relu(self.enc2(x))
        x = F.relu(self.enc3(x))
        x = F.relu(self.enc4(x))
        x = F.relu(self.enc5(x))

        x = F.relu(self.dec1(x))
        x = F.relu(self.dec2(x))
        x = F.relu(self.dec3(x))
        x = F.relu(self.dec4(x))
        x = F.relu(self.dec5(x))
        return x

net = Autoencoder()


From the Autoencoder class, we can see that 784 features are passed through a series of transformations and are converted to 16 features.

The transformations (in_features to out_features) for each layer are:

784 to 256
256 to 128
128 to 64
64 to 32
32 to 16


Why do we perform this sequence of operations? For example, why don't we perform the following sequence of operations instead?

784 to 256
256 to 128


Or maybe

784 to 512
512 to 256
256 to 128


Or maybe just encode in two layers:

784 to 16


Does the reduction of the dimensions over multiple layers (instead of a single layer) allow more details to be stored within the final representation? For example, if we used only the transformation $784 \rightarrow 16$, may this cause some detail not to be encoded? If so, why is this the case?

Well yes. Reduction of dimensions over multiple layers allow the model to learn more information than it would otherwise. You could do an experiment with it too; change the autoencoder to be much shallower and see what representation it learns!

In any case, think of it in simpler terms: Let's say you have a regression system. As opposed to a simple linear transformation like y = a0 + a1.x, a regression that uses higher degree polynomials to approximate the task has much more approximation power, for a lack of better terms. Which is why they tend to overfit easily; they're approximating the training data too perfectly and thus have trouble generalizing later. However, as a rule of thumb, an overfitting model may be considered as a more preferable scenario than an underfitting one: because you can regularize the overfitting model to generalize better, but the underfitting model may simply not be capable of approximating your task.

Now let's think of a deep neural network. At it's core, a neural network is also a highly complicated function that helps you approximate a task. If you have a linear layer that maps 784 input features to 16 output features, it's basically doing a simple transformation: ReLU(X.W+B). But if you're passing through multiple layers, your transformation becomes something like ReLU((ReLU((ReLU(X.W1+B1)).W2+B2)).W3+B3) and so on. It's a much more complex function with higher approximation power, which means it's liable to better approximate your task, and learn a better representation. This concept holds true for Deep Learning in general, and even truer for convolutional layers in particular. You'll notice most deep learning model architectures are basically layers or blocks repeatedly stacked one after another.
