e and then you can create a student UNet from the teacher UNet. The student UNet is updated by the optimizer during training.   Copied teacher_unet = UNet2DConditionModel.from_pretrained(
    args.pretrained_teacher_model, subfolder="unet", revision=args.teacher_revision
)

unet = UNet2DConditionModel(**teacher_unet.config)
unet.load_state_dict(teacher_unet.state_dict(), strict=False)
unet.train() Now you can create the optimizer to update the 