
	
When performing inference, you can merge the trained LoRA weights with the frozen pre-trained model weights, to interpolate between the original model’s inference result (as if no fine-tuning had occurred) and the fully fine-tuned version.
You can adjust the merging ratio with a parameter called α (alpha) in the paper, or scale in our implementation. You can tweak it with the following code, that passes scale as cross_attention_kwargs in the