Pytorch, backprop and composite models
Just a quick check for a question I have.

I want to build a model that generates its output based on two models F and G like so.

y = G(F(x))

where x is of course the input, and y the output.

However, first I want to update the weights of the F(x) and then later update the weights of both F and G based on the value of y.

I understand that pytorch offers a way to specify your own backprop method.. but since my "method" seems to build out of basic components, could it be that I can do this with a standard solution?

My thoughts would be that I need separate optimizer/loss for the F and G objects. But in addition to that, also some update functionality for the composite model G(F()). Can anyone confirm this as well?

If as you suggest, the optimizers and losses for F and G can be separated, then I don't think that it will be necessary to implement any different update functionalities since you can specify the set of parameters for each optimizer, e.g.

optimizer_F = optim.SGD(F.parameters(),...)
optimizer_G = optim.SGD(G.parameters(),...)


then when you call optimizer_F.step() it will only update the parameters of F and similarly optimizer_G.step() will only update the parameters of G.
