pt begins by adding the new LoRA weights to the attention layers. This involves correctly configuring the weight size for each block in the UNet. You’ll see the rank parameter is used to create the LoRAAttnProcessor:   Copied lora_attn_procs = {}
for name in unet.attn_processors.keys():
    cross_attention_dim = None if name.endswith("attn1.processor") else unet.config.cross_attention_dim
    if name.startswith("mid_block"):
        hidden_size