Parameters for Optimizer in ShuffleNet
In ShuffleNet the parameters that send to the optimizer first pass from the following function which makes a dictionary of weights:
def get_parameters(model):
    group_no_weight_decay = []
    group_weight_decay = []
    for pname, p in model.named_parameters():
        if pname.find('weight') &gt;= 0 and len(p.size()) &gt; 1:
            # print('include ', pname, p.size())
            group_weight_decay.append(p)
        else:
            # print('not include ', pname, p.size())
            group_no_weight_decay.append(p)
    assert len(list(model.parameters())) == len(group_weight_decay) + len(group_no_weight_decay)
    groups = [dict(params=group_weight_decay), dict(params=group_no_weight_decay, weight_decay=0.)]
    return groups

and then:
optimizer = optim.Adam(get_parameters(model), lr=0.01)

But what is the difference between this function and just using model.parameters  instead of get_parameters(model)?

With model.parameters() yo get all the parameters of the model in a single &quot;group&quot;, and thus all hyper parameters of the optimizer are the same for all model.parameters().
In contrast, get_parameters() groups model.parameters() into two groups: group_weight_decay and group_no_weight_decay. As the names suggest, for the parameters of the second group the optimizer sets the weight_decay hyper parameter to zero.
