ipeline.from_pretrained(base_model, torch_dtype=torch.float16).
This is especially useful when you don’t want to hardcode the base model identifier during initializing the StableDiffusionPipeline.
Inference for DreamBooth training remains the same. Check
this section for more details. 

Known limitations 
	
Currently, we only support LoRA for the attention layers of UNet2DConditionModel.
