Update README.md
Browse files
README.md
CHANGED
@@ -33,54 +33,73 @@ These are LoRA adaption weights for stable-diffusion-v1-5/stable-diffusion-v1-5.
|
|
33 |
#### How to use
|
34 |
|
35 |
```python
|
36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
37 |
```
|
38 |
|
39 |
#### Limitations and bias
|
40 |
|
41 |
[TODO: provide examples of latent issues and potential remediations]
|
42 |
|
43 |
-
## Training details
|
44 |
-
|
45 |
-
# Training Details - Stable Diffusion LoRA
|
46 |
|
47 |
# Dataset
|
48 |
-
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
|
53 |
|
54 |
# Model
|
55 |
-
|
56 |
-
|
57 |
-
|
58 |
-
|
59 |
|
60 |
# Preprocessing
|
61 |
-
|
62 |
-
|
63 |
-
|
64 |
-
|
65 |
|
66 |
# Training Configuration
|
67 |
-
|
68 |
-
|
69 |
-
|
70 |
-
|
71 |
-
|
72 |
-
|
73 |
-
|
74 |
-
|
75 |
|
76 |
# Validation
|
77 |
-
|
78 |
-
|
79 |
-
|
80 |
-
|
81 |
|
82 |
# Model Output
|
83 |
-
|
84 |
-
|
85 |
-
|
86 |
-
|
|
|
33 |
#### How to use
|
34 |
|
35 |
```python
|
36 |
+
import torch
|
37 |
+
import matplotlib.pyplot as plt
|
38 |
+
from diffusers import DiffusionPipeline
|
39 |
+
|
40 |
+
# Load the model and move it to GPU (CUDA)
|
41 |
+
pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5").to("cuda")
|
42 |
+
|
43 |
+
# Load the fine-tuned LoRA weights
|
44 |
+
pipe.load_lora_weights("Bhaskar009/SD_1.5_LoRA")
|
45 |
+
|
46 |
+
# Define a Naruto-themed prompt
|
47 |
+
prompt = "A detailed anime-style portrait of Naruto Uzumaki, wearing his Hokage cloak, standing under a bright sunset, ultra-detailed, cinematic lighting, 8K"
|
48 |
+
|
49 |
+
# Generate the image
|
50 |
+
image = pipe(prompt).images[0]
|
51 |
+
|
52 |
+
# Display the image using matplotlib
|
53 |
+
plt.figure(figsize=(6, 6))
|
54 |
+
plt.imshow(image)
|
55 |
+
plt.axis("off") # Hide axes for a clean view
|
56 |
+
plt.show()
|
57 |
+
|
58 |
```
|
59 |
|
60 |
#### Limitations and bias
|
61 |
|
62 |
[TODO: provide examples of latent issues and potential remediations]
|
63 |
|
64 |
+
## Training details - Stable Diffusion LoRA
|
|
|
|
|
65 |
|
66 |
# Dataset
|
67 |
+
|
68 |
+
-The model was trained using the 'lambdalabs/naruto-blip-captions' dataset.
|
69 |
+
-This dataset consists of Naruto character images with BLIP-generated captions.
|
70 |
+
-It provides a diverse set of characters, poses, and backgrounds,
|
71 |
+
-making it suitable for fine-tuning Stable Diffusion on anime-style images.
|
72 |
|
73 |
# Model
|
74 |
+
|
75 |
+
-Base Model: Stable Diffusion v1.5 (stable-diffusion-v1-5/stable-diffusion-v1-5)
|
76 |
+
-Fine-tuning Method: LoRA (Low-Rank Adaptation)
|
77 |
+
-Purpose: Specializing Stable Diffusion to generate Naruto-style anime characters.
|
78 |
|
79 |
# Preprocessing
|
80 |
+
|
81 |
+
- Images were resized to 512x512 resolution.
|
82 |
+
- Center cropping was applied to maintain aspect ratio.
|
83 |
+
- Random flipping was used as a data augmentation technique.
|
84 |
|
85 |
# Training Configuration
|
86 |
+
|
87 |
+
-Batch Size: 1
|
88 |
+
-Gradient Accumulation Steps: 4 # Simulates a larger batch size
|
89 |
+
-Gradient Checkpointing: Enabled # Reduces memory consumption
|
90 |
+
-Max Training Steps: 800
|
91 |
+
-Learning Rate: 1e-5 (constant schedule, no warmup)
|
92 |
+
-Max Gradient Norm: 1 # Prevents gradient explosion
|
93 |
+
-Memory Optimization: xFormers enabled for efficient attention computation
|
94 |
|
95 |
# Validation
|
96 |
+
|
97 |
+
- A validation prompt "A Naruto character" was used.
|
98 |
+
- 4 validation images were generated during training.
|
99 |
+
- Model checkpoints were saved every 500 steps.
|
100 |
|
101 |
# Model Output
|
102 |
+
|
103 |
+
- The fine-tuned LoRA model was saved to "sd-naruto-model".
|
104 |
+
- The model was pushed to the Hugging Face Hub:
|
105 |
+
- Repository: Bhaskar009/SD_1.5_LoRA
|