How to initialize mean and variance of Pytorch BatchNorm2d?
I’m transforming a TensorFlow model to Pytorch. And I’d like to initialize the mean and variance of BatchNorm2d using TensorFlow model.
I’m doing it in this way:

bn.running_mean = torch.nn.Parameter(torch.Tensor(TF_param))


And I get this error:

RuntimeError: the derivative for 'running_mean' is not implemented


But is works for bn.weight and bn.bias. Is there any way to initialize the mean and variance using my pre-trained Tensorflow model? Is there anything like moving_mean_initializer and moving_variance_initializer in Pytorch?

Thanks!

The running mean and variance of a batch norm layer are not nn.Parameters, but rather a buffer of the layer.

I think you can simply assign a torch.tensor, no need to wrap a nn.Parameter around it.
