NoisyNet DQN with default parameters not exploring
I think the problem is that you are not defining the weights and biases as parameters. So, when you backpropagate, they are not modified.

These lines should do the trick:

self.weight_mu = Parameter(torch.Tensor(out_features, in_features))
self.weight_sigma = Parameter(torch.Tensor(out_features, in_features))
self.bias_mu = Parameter(torch.Tensor(out_features))
self.bias_sigma = Parameter(torch.Tensor(out_features))


In case you are not familiar, the Parameter class must be imported from torch:

from torch.nn.parameter import Parameter