how pytorch nn.module save submodule
I have some question about how pytorch nn.module works

import torch
import torch.nn as nn



class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.sub_module = nn.Linear(10, 5)
        self.value = 3

net = Net()
print(net.__dict__)


output 

{'_modules': OrderedDict([('sub_module', Linear (10 -&gt; 5))]),  'value': 3, ...}


I know that every attribute of a class should be stored in __dict__, why value(a int value) is in it, but sub_module(a nn.Module) is not, instead, sub_module is stored in _modules

I read the code of nn.Module implementation, but I didn't figure that out. do anyone have any ideas?

thank you !!

I will try to keep it simple. 

Every time you create a new item in the class Net for instance: self.sub_module = nn.Linear(10, 5) it calls the method __setattr__ of its parent class, in this case nn.Module. Then, inside __setattr__ method, the parameters are stored to the dict they belong. In this case since nn.Linear is a module, it is stored to the _modules dict.

Here is the piece of code that does this inside the Module class  https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/module.py#L389
