Create a simple PyTorch neural network with a normalized weights
I want to create a simple PyTorch neural network with the sum of its weights equal to 1. To understand my question here is a to give an example:


You can simply normalize by the sum of all initialized weights:
&gt;&gt;&gt; layer = nn.Linear(4, 1, bias=False)
&gt;&gt;&gt; layer.weight
Parameter containing:
tensor([[-0.2565,  0.4753, -0.1129,  0.2327]], requires_grad=True)

Normalize layer.weight:
&gt;&gt;&gt; layer.weight.data /= layer.weight.data.sum()

Then:
&gt;&gt;&gt; layer.weight
Parameter containing:
tensor([[-0.7573,  1.4034, -0.3333,  0.6872]], requires_grad=True)

