PyTorch Conv2D returns non-zero output for an input tensor of zeros?
If you input an array consisting only of zeros to a Conv2D layer, the output should also consist only of zeros. In TensorFlow, this is the case. But, in PyTorch, it isn't. Here is some very simple sample Python code to demonstrate this. Why does PyTorch output non-zero numbers in this situation?
import torch
import numpy as np

image = np.zeros((3,3,3), dtype=np.float32)
batch = np.asarray([image])

a = torch.nn.Conv2d(3,3,1)
b = a(torch.tensor(batch).permute(0,3,1,2))

print(b.permute(0,2,3,1))


Unlike Tensorflow, PyTorch initializes the bias with non-zero values (see the source-code):
def reset_parameters(self) -&gt; None:
    init.kaiming_uniform_(self.weight, a=math.sqrt(5))
    if self.bias is not None:
        fan_in, _ = init._calculate_fan_in_and_fan_out(self.weight)
        bound = 1 / math.sqrt(fan_in)
        init.uniform_(self.bias, -bound, bound)

