Why does this tensor.unfold method in this example add a dimension?
I am familiarizing myself with the Pytorch unfold method from https://pytorch.org/docs/stable/tensors.html#torch.Tensor.unfold

I looked at their example which is

&gt;&gt;&gt; x = torch.arange(1., 8)
&gt;&gt;&gt; x
tensor([ 1.,  2.,  3.,  4.,  5.,  6.,  7.])
&gt;&gt;&gt; x.unfold(0, 2, 1)
tensor([[ 1.,  2.],
        [ 2.,  3.],
        [ 3.,  4.],
        [ 4.,  5.],
        [ 5.,  6.],
        [ 6.,  7.]])


I understand above that when we unfold in dimension 0, we take chunks of size 2 at a time with stride 1 and therefore, the result is an arrangement of different chunks, which are [1., 2.], [2., 3.] and so on. As we have 6 chunks at the end, the chunks will be put together and the final shape is (6,2).

However, I have another example I ran as shown below.

In [115]: s = torch.arange(20).view(1,10,2)

In [116]: s
Out[116]:
tensor([[[ 0,  1],
         [ 2,  3],
         [ 4,  5],
         [ 6,  7],
         [ 8,  9],
         [10, 11],
         [12, 13],
         [14, 15],
         [16, 17],
         [18, 19]]])

In [117]: s.unfold(0,1,1)
Out[117]:
tensor([[[[ 0],
          [ 1]],

         [[ 2],
          [ 3]],

         [[ 4],
          [ 5]],

         [[ 6],
          [ 7]],

         [[ 8],
          [ 9]],

         [[10],
          [11]],

         [[12],
          [13]],

         [[14],
          [15]],

         [[16],
          [17]],

         [[18],
          [19]]]])

In [119]: s.unfold(0,1,1).shape
Out[119]: torch.Size([1, 10, 2, 1])


So you see my original tensor was of shape (1,10,2) and I asked for an unfolding operation with parameters s.unfold(0, 1, 1).

Going by original understanding from the previous example, I assumed this means in the dimension 0, we take 1 chunk at a time with stride 1. Thus, as we have go into dimension 0, we see that we have only one chunk of size (10,2). So the output should have just taken this chunk and may be it should have just added a dimension to wrap this chunk and given me an output of size (1, 10, 2). 

However, it gives me an output of size (1, 10, 2, 1). Why does it have an extra dimension at the last? Can someone elaborate intuitively please?

The documentation states:


  An additional dimension of size size is appended in the returned tensor.


where size is the size of the chunks you specified (second argument). By definition, it always adds an additional dimension, which makes it consistent no matter what size you choose. Just because a dimension has size 1, doesn't mean it should be omitted automatically.

Regarding the intuition behind this, let's consider that instead of returning a tensor where the last dimension represents the chunks, we create a list of all chunks. For simplicity, we'll limit it to the first dimension with a step of 1.

import torch
from typing import List


def list_chunks(tensor: torch.Tensor, size: int) -&gt; List[torch.Tensor]:
    chunks = []
    for i in range(tensor.size(0) - size + 1):
        chunks.append(tensor[i : i + size])
    return chunks


x = torch.arange(1.0, 8)
s = torch.arange(20).view(1, 10, 2)

# As expected, a list with 6 elements, as there are 6 chunks.
list_chunks(x, 2)
# =&gt; [tensor([1., 2.]),
#     tensor([2., 3.]),
#     tensor([3., 4.]),
#     tensor([4., 5.]),
#     tensor([5., 6.]),
#     tensor([6., 7.])]

# The list has only a single element, as there is only a single chunk.
# But it's still a list.
list_chunks(s, 1)
# =&gt; [tensor([[[ 0,  1],
#              [ 2,  3],
#              [ 4,  5],
#              [ 6,  7],
#              [ 8,  9],
#              [10, 11],
#              [12, 13],
#              [14, 15],
#              [16, 17],
#              [18, 19]]])]


I've deliberately included type annotations to make it clearer what we are expecting from the function. If there is only a single chunk, it will be a list with one element, as it is always a list of chunks.

You were expecting a different behaviour, namely when there is a single chunk, you want the single chunk instead of a list. Which would change the implementation as follows.

from typing import List, Union


def list_chunks(tensor: torch.Tensor, size: int) -&gt; Union[List[torch.Tensor], torch.Tensor]:
    chunks = []
    for i in range(tensor.size(0) - size + 1):
        chunks.append(tensor[i : i + size])
    # If it's a single chunk, return just the chunk itself
    if len(chunks) == 1:
        return chunks[0]
    else:
        return chunks


With that change, anyone that uses this function, now needs to take two cases into consideration. If you don't distinguish between a list and a single chunk (tensor), you will get unexpected results, e.g. looping over the chunks would instead loop over the first dimension of the tensor.

The programmatically intuitive approach is to always return a list of chunks and torch.unfold does the same, but instead of a list of chunks, it's a tensor where the last dimension can be seen as the listing of the chunks.
