(relu1): ReLU()
     (fc2): LazyLinear(in_features=0, out_features=1, bias=True)
     (relu2): ReLU()
   )



performs a dry run to initialize the network's lazy modules
lazy_mlp(torch.ones(10,10).cuda())
after initialization, LazyLinear modules become regular Linear modules
lazy_mlp
   LazyMLP(
     (fc1): Linear(in_features=10, out_features=10, bias=True)
     (relu1): ReLU()
     (fc2): Linear(in_features=10, out_features=1, bias=True)
     (relu2): ReLU()
   )
attaches an optimizer, since parameters can now be used as usual
optim = torch.optim.SGD(mlp.parameters(), lr=0.01)



A final caveat when using lazy modules is that the order of
   initialization of a network's parameters may change, since the lazy
   modules are always initialized after other modules. For example, if
   the LazyMLP class defined above had a "torch.nn.LazyLinear" module
   first and then a regular "torch.nn.Linear" second, the second