Pytorch how to reshape/reduce the number of filters without altering the shape of the individual filters
With a 3D tensor of shape (number of filters, height, width), how can one reduce the number of filters with a reshape which keeps the original filters together as whole blocks?
Assume the new size has dimensions chosen such that a whole number of the original filters can fit side by side in one of the new filters. So an original size of (4, 2, 2) can be reshaped to (2, 2, 4).
A visual explanation of the side by side reshape where you see the standard reshape will alter the individual filter shapes:

I have tried various pytorch functions such as gather and select_index but not found a way to get to the end result in a general manner (i.e. works for different numbers of filters and different filter sizes).
I think it would be easier to rearrange the tensor values after performing the reshape but could not get a tensor of the pytorch reshaped form:
[[[1,2,3,4],
  [5,6,7,8]],
 
 [[9,10,11,12],
  [13,14,15,16]]]

to:
[[[1,2,5,6],
  [3,4,7,8]],

 [[9,10,13,14],
  [11,12,15,16]]]

for completeness, the original tensor before reshaping:
[[[1,2],
  [3,4]],
 
 [[5,6],
  [7,8]],

 [[9,10],
  [11,12]],

 [[13,14],
  [15,16]]]


Another option is to construct a list of parts and concatenate them
x = torch.arange(4).reshape(4, 1, 1).repeat(1, 2, 2)
y = torch.cat([x[i::2] for i in range(2)], dim=2)

print('Before\n', x)
print('After\n', y)

which gives
Before
 tensor([[[0, 0],
         [0, 0]],

        [[1, 1],
         [1, 1]],

        [[2, 2],
         [2, 2]],

        [[3, 3],
         [3, 3]]])
After
 tensor([[[0, 0, 1, 1],
         [0, 0, 1, 1]],

        [[2, 2, 3, 3],
         [2, 2, 3, 3]]])

Or a little more generally we could write a function that takes groups of neighbors along a source dimension and concatenates them along a destination dimension
def group_neighbors(x, group_size, src_dim, dst_dim):
    assert x.shape[src_dim] % group_size == 0
    return torch.cat([x[[slice(None)] * (src_dim) + [slice(i, None, group_size)] + [slice(None)] * (len(x.shape) - (src_dim + 2))] for i in range(group_size)], dim=dst_dim)


x = torch.arange(4).reshape(4, 1, 1).repeat(1, 2, 2)
# read as &quot;take neighbors in groups of 2 from dimension 0 and concatenate them in dimension 2&quot;
y = group_neighbors(x, group_size=2, src_dim=0, dst_dim=2)

print('Before\n', x)
print('After\n', y)

