How to add parameters from self.some_dictionary_of_modules to self.parameters?
Consider this simple example:

import torch

class MyModule(torch.nn.Module):
    def __init__(self):
        super(MyModule, self).__init__()
        self.conv_0=torch.nn.Conv2d(3,32,3,stride=1,padding=0)
        self.blocks=torch.nn.ModuleList([
            torch.nn.Conv2d(3,32,3,stride=1,padding=0),
            torch.nn.Conv2d(32,64,3,stride=1,padding=0)])

        #the problematic part
        self.dict_block={"key_1": torch.nn.Conv2d(64,128,3,1,0),
                "key_2": torch.nn.Conv2d(56,1024,3,1,0)}

if __name__=="__main__":
    my_module=MyModule()
    print(my_module.parameters)


The output I get here is (notice that the parmeters from self.dict_block are missing) 

&lt;bound method Module.parameters of MyModule(
  (conv_0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1))
  (blocks): ModuleList(
    (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1))
    (1): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1))
  )
)&gt;


This means that if I want the parameters of self.dict_block to be optimised, I will need to use something  like 

my_optimiser.add_param_group({"params": params_from_self_dict})


before using my optimiser. However, I think there might be a more straight-forward alternative that will add the parameters of self.dict_block to the parameters of the my_module_object. Something that comes close is nn.Parameter(...) as explained here, but this requires the input to be a tensor.

Found the answer. Posting it in case someone runs into the same problem:

Looking into &lt;torch_install&gt;/torch/nn/modules/container.py it looks like there is a class torch.nn.ModuleDict that just does that. So in the example I gave in the question, the solution would be:

self.dict_block=torch.nn.ModuleDict({"key_1": torch.nn.Conv2d(64,128,3,1,0),
            "key_2": torch.nn.Conv2d(56,1024,3,1,0)})

