Update README.md
Browse files
README.md
CHANGED
@@ -1,8 +1,5 @@
|
|
1 |
-
---
|
2 |
-
library_name: diffusers
|
3 |
-
---
|
4 |
# ControlNet Standard Lineart for SDXL
|
5 |
-
SDXL has perfect content generation functions and amazing LoRa performance, but its ControlNet is always its drawback,
|
6 |
|
7 |
|
8 |
**This model attempts to fill the insufficiency of the ControlNet for SDXL to lower the requirements for SDXL to personal users.**
|
@@ -53,4 +50,23 @@ image = pipe(
|
|
53 |
|
54 |
## Result:
|
55 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
56 |
|
|
|
|
|
|
|
|
|
1 |
# ControlNet Standard Lineart for SDXL
|
2 |
+
SDXL has perfect content generation functions and amazing LoRa performance, but its ControlNet is always its drawback, filtering out most of the users. Based on the computational power constraints of personal GPU, one cannot easily train and tune a perfect ControlNet model.
|
3 |
|
4 |
|
5 |
**This model attempts to fill the insufficiency of the ControlNet for SDXL to lower the requirements for SDXL to personal users.**
|
|
|
50 |
|
51 |
## Result:
|
52 |
|
53 |
+

|
54 |
+
|
55 |
+
## Note:
|
56 |
+
|
57 |
+
1. Loading custom datasets through HuggingFace needs to modify the script to realize full automation. In the ***train_controlnet_sdxl.py***, we need to modify line 650 to:
|
58 |
+
```python
|
59 |
+
if args.train_data_dir is not None:
|
60 |
+
dataset = load_dataset(
|
61 |
+
args.train_data_dir,
|
62 |
+
cache_dir=args.cache_dir,
|
63 |
+
trust_remote_code=True,
|
64 |
+
)
|
65 |
+
```
|
66 |
+
As for the dataset, we need to organize the structure as demonstrated in the [dataset_example](https://civitai.com/articles/2078/play-in-control-controlnet-training-setup-guide), and change the script to:
|
67 |
+
```
|
68 |
+
--train_data_dir="/path/to/your/dataset_example"
|
69 |
+
```
|
70 |
+
|
71 |
+
2. Based on the experiment, sometimes this ControlNet cannot understand colorization very well on the xl-base-1.0. However, it can capture the line perfectly. So I suspect the miss colorization happened on the base model I chose. More experiments are needed.
|
72 |
|