Find all ReLU layer in a torchvision model
After I fetch a pre-trained model from torchvision.models, I want all the ReLU instance to register_backward_hook(f),which is like this:

for pos, module in self.model.features._modules.items():
    for sub_module in module:
        if isinstance(module, ReLU):
            module.register_backward_hook(f)


The problems for me is how to find all ReLU in a model. For densenet161, the ReLU exists not only in model.features._modules but also in self-defined dense layer, eg. model.features._modules['denseblock1'][0]. For  resnet151, the ReLU exists in model._modules and its self-define layer, eg model._modules['layer1'].

Is there any way to find all ReLU inside a model?

A more elegant way to iterate over all components of a model is using modules()
method:

from torch import nn

for module in self.model.modules():
  if isinstance(module, nn.ReLU):
    module.register_backward_hook(f)


If you do not want to get all sub-modules, only the immediate ones, you may consider using children() method instead of modules(). You can also get the name of the sub module using named_modules() method.
