File size: 5,426 Bytes
c993df5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 |
---
frameworks:
- Pytorch
license: apache-2.0
tasks:
- efficient-diffusion-tuning
---
<p align="center">
<h2 align="center">clay_style_edit</h2>
<p align="center">
<br>
<a href="https://github.com/modelscope/scepter/"><img src="https://img.shields.io/badge/powered by-scepter-6FEBB9.svg"></a>
<br>
</p>
## Model Introduction
Transfer images into clay style
## Model Parameters
<table>
<thead>
<tr>
<th rowspan="2">Base Model</th>
<th rowspan="2">Tuner Type</th>
<th colspan="4">Training Parameters</th>
</tr>
<tr>
<th>Batch Size</th>
<th>Epochs</th>
<th>Learning Rate</th>
<th>Resolution</th>
</tr>
</thead>
<tbody align="center">
<tr>
<td rowspan="8">EDIT</td>
<td>LORA</td>
<td>1</td>
<td>50</td>
<td>0.0001</td>
<td>[512, 512]</td>
</tr>
</tbody>
</table>
<table>
<thead>
<tr>
<th>Data Type</th>
<th>Data Space</th>
<th>Data Name</th>
<th>Data Subset</th>
</tr>
</thead>
<tbody align="center">
<tr>
<td>Image Edit Generation</td>
<td></td>
<td>clay-v1-20240527_16_06_41</td>
<td>default</td>
</tr>
</tbody>
</table>
## Model Performance
Given the input "Convert this image into clay style," the following image may be generated:
## Model Usage
### Command Line Execution
* Run using Scepter's SDK, taking care to use different configuration files in accordance with the different base models, as per the corresponding relationships shown below
<table>
<thead>
<tr>
<th rowspan="2">Base Model</th>
<th rowspan="1">LORA</th>
<th colspan="1">SCE</th>
<th colspan="1">TEXT_LORA</th>
<th colspan="1">TEXT_SCE</th>
</tr>
</thead>
<tbody align="center">
<tr>
<td rowspan="8">SD1.5</td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/examples/generation/stable_diffusion_1.5_512_lora.yaml">lora_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/scedit/t2i/sd15_512_sce_t2i_swift.yaml">sce_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/examples/generation/stable_diffusion_1.5_512_text_lora.yaml">text_lora_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/scedit/t2i/stable_diffusion_1.5_512_text_sce.yaml">text_sce_cfg</a></td>
</tr>
</tbody>
<tbody align="center">
<tr>
<td rowspan="8">SD2.1</td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/examples/generation/stable_diffusion_2.1_768_lora.yaml">lora_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/scedit/t2i/sd21_768_sce_t2i_swift.yaml">sce_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/examples/generation/stable_diffusion_2.1_768_text_lora.yaml">text_lora_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/scedit/t2i/sd21_768_text_sce_t2i_swift.yaml">text_sce_cfg</a></td>
</tr>
</tbody>
<tbody align="center">
<tr>
<td rowspan="8">SDXL</td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/examples/generation/stable_diffusion_xl_1024_lora.yaml">lora_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/scedit/t2i/sdxl_1024_sce_t2i_swift.yaml">sce_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/examples/generation/stable_diffusion_xl_1024_text_lora.yaml">text_lora_cfg</a></td>
<td><a href="https://github.com/modelscope/scepter/blob/main/scepter/methods/scedit/t2i/sdxl_1024_text_sce_t2i_swift.yaml">text_sce_cfg</a></td>
</tr>
</tbody>
</table>
* Running from Source Code
```shell
git clone https://github.com/modelscope/scepter.git
cd scepter
pip install -r requirements/recommended.txt
PYTHONPATH=. python scepter/tools/run_inference.py
--pretrained_model {this model folder}
--cfg {lora_cfg} or {sce_cfg} or {text_lora_cfg} or {text_sce_cfg}
--prompt 'Convert this image into clay style'
--save_folder 'inference'
```
* Running after Installing Scepter (Recommended)
```shell
pip install scepter
python -m scepter/tools/run_inference.py
--pretrained_model {this model folder}
--cfg {lora_cfg} or {sce_cfg} or {text_lora_cfg} or {text_sce_cfg}
--prompt 'Convert this image into clay style'
--save_folder 'inference'
```
### Running with Scepter Studio
```shell
pip install scepter
# Launch Scepter Studio
python -m scepter.tools.webui
```
* Refer to the following guides for model usage.
(video url)
## Model Reference
If you wish to use this model for your own purposes, please cite it as follows.
```bibtex
@misc{clay_style_edit,
title = {clay_style_edit, {MODEL_URL}},
author = {{USER_NAME}},
year = {2024}
}
```
This model was trained using [Scepter Studio](https://github.com/modelscope/scepter); [Scepter](https://github.com/modelscope/scepter)
is an algorithm framework and toolbox developed by the Alibaba Tongyi Wanxiang Team. It provides a suite of tools and models for image generation, editing, fine-tuning, data processing, and more. If you find our work beneficial for your research,
please cite as follows.
```bibtex
@misc{scepter,
title = {SCEPTER, https://github.com/modelscope/scepter},
author = {SCEPTER},
year = {2023}
}
```
|