My PyTorch Conv1d with an identity kernel does not produce the same output as the input
I have:

        print('\ninp', inp.min(), inp.mean(), inp.max())
        print(inp)
        out = self.conv1(inp)
        print('\nout1', out.min(), out.mean(), out.max())
        print(out)
        quit()


My min, mean and max for my inp is: inp tensor(9.0060e-05) tensor(0.1357) tensor(2.4454)

For my output, I have: out1 tensor(4.8751, grad_fn=&lt;MinBackward1&gt;) tensor(21.8416, grad_fn=&lt;MeanBackward0&gt;) tensor(54.9332, grad_fn=&lt;MaxBackward1&gt;)

My self.conv1 is:

        self.conv1 = torch.nn.Conv1d(
            in_channels=161,
            out_channels=161,
            kernel_size=11,
            stride=1,
            padding=5)
        self.conv1.weight.data = torch.zeros(self.conv1.weight.data.size())
        self.conv1.weight.data[:, :, 5] = 1.0
        self.conv1.bias.data = torch.zeros(self.conv1.bias.data.size())


So my weights look like: tensor([0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0.])

So if I understand how convolution works, this should produce the same output. But it doesn't.

What am I doing wrong?

Always try to provide a Minimal, Reproducible Example.



It shouldn't. You are probably forgetting the summation. As stated in the docs:


  In the simplest case, the output value of the layer with input size
  (N, C_in, L) and output (N, C_out, L_out) can be precisely
  described as:
  
  
  where ⋆ is the valid cross-correlation operator, N is a batch size, C denotes a number of channels, L is a length of signal sequence.


Notice that, in your example, the mean after the conv (i.e., 21.8416) is approx. 161 times the mean before (i.e., 161 * 0.1357), and this is not a coincidence. Notice the same happening in the code below:

import torch
torch.manual_seed(0)

# define the fake input data
x = torch.rand(1, 3, 5)
# &gt;&gt;&gt; x
# tensor([[[0.4963, 0.7682, 0.0885, 0.1320, 0.3074],
#          [0.6341, 0.4901, 0.8964, 0.4556, 0.6323],
#          [0.3489, 0.4017, 0.0223, 0.1689, 0.2939]]])

# define the conv
conv1 = torch.nn.Conv1d(3, 3, kernel_size=5, stride=1, padding=2)
conv1.weight.data = torch.zeros(conv1.weight.data.size())
conv1.weight.data[:, :, 2] = 1.0
conv1.bias.data = torch.zeros(conv1.bias.data.size())

# print mean before
print(x.mean())
# tensor(0.4091)

# print mean after
print(conv1(x).mean())
# tensor(1.2273, grad_fn=&lt;MeanBackward0&gt;)


See? After the conv, the mean is 3 times the original one.

As @jodag said, if you want an identity, you can do like this:

import torch
torch.manual_seed(0)

# define the fake input data
x = torch.rand(1, 3, 5)
# &gt;&gt;&gt; x
# tensor([[[0.4963, 0.7682, 0.0885, 0.1320, 0.3074],
#          [0.6341, 0.4901, 0.8964, 0.4556, 0.6323],
#          [0.3489, 0.4017, 0.0223, 0.1689, 0.2939]]])

# define the conv
conv1 = torch.nn.Conv1d(3, 3, kernel_size=5, stride=1, padding=2)
torch.nn.init.zeros_(conv1.weight)
torch.nn.init.zeros_(conv1.bias)
# set identity kernel
conv1.weight.data[:, :, 2] = torch.eye(3, 3)

# print mean before
print(x.mean())
# tensor(0.4091)

# print mean after
print(conv1(x).mean())
# tensor(0.4091, grad_fn=&lt;MeanBackward0&gt;)

