Update targets in classification images
Why are we updating targets in the implementation of bayesian cnn with mc dropout here?
https://github.com/sungyubkim/MCDO/blob/master/Bayesian_CNN_with_MCDO.ipynb?fbclid=IwAR18IMLcdUUp90TRoYodsJS7GW1smk-KGYovNpojn8LtRhDQckFI_gnpOYc
def update_target(target, original, update_rate):
        for target_param, param in zip(target.parameters(), original.parameters()):
            target_param.data.copy_((1.0 - update_rate) * target_param.data + update_rate*param.data)


The implementation you have referred to is a data parallel one.
Which means, the author intends to train multiple networks with the same architecture but different hyper-parameters.
Although in an unconventional way, this is what update_target does:
update_target(net_test, net, 0.001)

It updates the net_test with a lower learning rate compared to net, but with the exact same parameter changes applied to original net, that is actually being trained. Only the change scales is different.
I am assuming that this is found to be useful in terms of computational efficiency, since only one of the networks are actually being &quot;trained&quot; during main training phase:
outputs = net(inputs)
loss = CE(outputs, labels)
loss.backward()
optimizer.step()

One less forward pass and one less backprop per step.
