tream performance with great efficiency. This is why, it’s common
to just add the LoRA weights to the attention layers of a model.
cloneofsimo was the first to try out LoRA training for Stable Diffusion in the popular lora GitHub repository.
LoRA allows us to achieve greater memory efficiency since the pretrained weights are kept frozen and only the LoRA weights are trained, thereby
allowing us to run fine-tuning on consumer GPUs like Tesla T4,