Bhaskar009 commited on
Commit
a6c1a67
·
verified ·
1 Parent(s): d8d5e14

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -33
README.md CHANGED
@@ -33,54 +33,73 @@ These are LoRA adaption weights for stable-diffusion-v1-5/stable-diffusion-v1-5.
33
  #### How to use
34
 
35
  ```python
36
- # TODO: add an example code snippet for running this diffusion pipeline
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  ```
38
 
39
  #### Limitations and bias
40
 
41
  [TODO: provide examples of latent issues and potential remediations]
42
 
43
- ## Training details
44
-
45
- # Training Details - Stable Diffusion LoRA
46
 
47
  # Dataset
48
- # ------------------------------------------
49
- # The model was trained using the 'lambdalabs/naruto-blip-captions' dataset.
50
- # This dataset consists of Naruto character images with BLIP-generated captions.
51
- # It provides a diverse set of characters, poses, and backgrounds,
52
- # making it suitable for fine-tuning Stable Diffusion on anime-style images.
53
 
54
  # Model
55
- # ------------------------------------------
56
- # Base Model: Stable Diffusion v1.5 (stable-diffusion-v1-5/stable-diffusion-v1-5)
57
- # Fine-tuning Method: LoRA (Low-Rank Adaptation)
58
- # Purpose: Specializing Stable Diffusion to generate Naruto-style anime characters.
59
 
60
  # Preprocessing
61
- # ------------------------------------------
62
- # - Images were resized to 512x512 resolution.
63
- # - Center cropping was applied to maintain aspect ratio.
64
- # - Random flipping was used as a data augmentation technique.
65
 
66
  # Training Configuration
67
- # ------------------------------------------
68
- # Batch Size: 1
69
- # Gradient Accumulation Steps: 4 # Simulates a larger batch size
70
- # Gradient Checkpointing: Enabled # Reduces memory consumption
71
- # Max Training Steps: 800
72
- # Learning Rate: 1e-5 (constant schedule, no warmup)
73
- # Max Gradient Norm: 1 # Prevents gradient explosion
74
- # Memory Optimization: xFormers enabled for efficient attention computation
75
 
76
  # Validation
77
- # ------------------------------------------
78
- # - A validation prompt "A Naruto character" was used.
79
- # - 4 validation images were generated during training.
80
- # - Model checkpoints were saved every 500 steps.
81
 
82
  # Model Output
83
- # ------------------------------------------
84
- # - The fine-tuned LoRA model was saved to "sd-naruto-model".
85
- # - The model was pushed to the Hugging Face Hub:
86
- # Repository: Bhaskar009/SD_1.5_LoRA
 
33
  #### How to use
34
 
35
  ```python
36
+ import torch
37
+ import matplotlib.pyplot as plt
38
+ from diffusers import DiffusionPipeline
39
+
40
+ # Load the model and move it to GPU (CUDA)
41
+ pipe = DiffusionPipeline.from_pretrained("stable-diffusion-v1-5/stable-diffusion-v1-5").to("cuda")
42
+
43
+ # Load the fine-tuned LoRA weights
44
+ pipe.load_lora_weights("Bhaskar009/SD_1.5_LoRA")
45
+
46
+ # Define a Naruto-themed prompt
47
+ prompt = "A detailed anime-style portrait of Naruto Uzumaki, wearing his Hokage cloak, standing under a bright sunset, ultra-detailed, cinematic lighting, 8K"
48
+
49
+ # Generate the image
50
+ image = pipe(prompt).images[0]
51
+
52
+ # Display the image using matplotlib
53
+ plt.figure(figsize=(6, 6))
54
+ plt.imshow(image)
55
+ plt.axis("off") # Hide axes for a clean view
56
+ plt.show()
57
+
58
  ```
59
 
60
  #### Limitations and bias
61
 
62
  [TODO: provide examples of latent issues and potential remediations]
63
 
64
+ ## Training details - Stable Diffusion LoRA
 
 
65
 
66
  # Dataset
67
+
68
+ -The model was trained using the 'lambdalabs/naruto-blip-captions' dataset.
69
+ -This dataset consists of Naruto character images with BLIP-generated captions.
70
+ -It provides a diverse set of characters, poses, and backgrounds,
71
+ -making it suitable for fine-tuning Stable Diffusion on anime-style images.
72
 
73
  # Model
74
+
75
+ -Base Model: Stable Diffusion v1.5 (stable-diffusion-v1-5/stable-diffusion-v1-5)
76
+ -Fine-tuning Method: LoRA (Low-Rank Adaptation)
77
+ -Purpose: Specializing Stable Diffusion to generate Naruto-style anime characters.
78
 
79
  # Preprocessing
80
+
81
+ - Images were resized to 512x512 resolution.
82
+ - Center cropping was applied to maintain aspect ratio.
83
+ - Random flipping was used as a data augmentation technique.
84
 
85
  # Training Configuration
86
+
87
+ -Batch Size: 1
88
+ -Gradient Accumulation Steps: 4 # Simulates a larger batch size
89
+ -Gradient Checkpointing: Enabled # Reduces memory consumption
90
+ -Max Training Steps: 800
91
+ -Learning Rate: 1e-5 (constant schedule, no warmup)
92
+ -Max Gradient Norm: 1 # Prevents gradient explosion
93
+ -Memory Optimization: xFormers enabled for efficient attention computation
94
 
95
  # Validation
96
+
97
+ - A validation prompt "A Naruto character" was used.
98
+ - 4 validation images were generated during training.
99
+ - Model checkpoints were saved every 500 steps.
100
 
101
  # Model Output
102
+
103
+ - The fine-tuned LoRA model was saved to "sd-naruto-model".
104
+ - The model was pushed to the Hugging Face Hub:
105
+ - Repository: Bhaskar009/SD_1.5_LoRA