How does pytorch L1-norm pruning works?
Lets see the result that I got first. This is one of a convolution layer of my model, and im only showing 11 filter's weight of it (11 3x3 filter with channel=1)
Left side is original weight Right side is Pruned weight
So I was wondering how does the &quot;TORCH.NN.UTILS.PRUNE.L1_UNSTRUCTURED&quot; works because by the pytorch website said, it prune the lowest L1-norm unit, but as far as I know, L1-norm pruning is a filter pruning method which prune the whole filter which use this equation to fine the lowest filter value instead of pruning single weight. So I'm a bit curious about how does this function actually works?
The following is my pruning code
parameters_to_prune = (
    (model.input_layer[0], 'weight'),
    (model.hidden_layer1[0], 'weight'),
    (model.hidden_layer2[0], 'weight'),
    (model.output_layer[0], 'weight')
)

prune.global_unstructured(
    parameters_to_prune,
    pruning_method=prune.L1Unstructured,
    amount = (pruned_percentage/100),
)


The nn.utils.prune.l1_unstructured utility does not prune the whole filter, it prunes individual parameter components as you observed in your sheet. That is components with the lower norm get masked.

Here is a minimal example as discussed in the comments below:
&gt;&gt;&gt; m = nn.Linear(10,1,bias=False)
&gt;&gt;&gt; m.weight = nn.Parameter(torch.arange(10).float())
&gt;&gt;&gt; prune.l1_unstructured(m, 'weight', .3)
&gt;&gt;&gt; m.weight
tensor([0., 0., 0., 3., 4., 5., 6., 7., 8., 9.], grad_fn=&lt;MulBackward0&gt;)

