Spaces:
Configuration error
Configuration error
Commit
·
5357c60
1
Parent(s):
dd95d8d
update
Browse files- README.md +39 -20
- assets/soely_edit/input.gif +3 -0
- assets/soely_edit/joint.gif +3 -0
- assets/soely_edit/left.gif +3 -0
- assets/soely_edit/right.gif +3 -0
- assets/vis/bear_weight.gif +3 -0
- assets/vis/cherry_weight.gif +3 -0
- assets/vis/edit.gif +3 -0
- assets/vis/spiderman_weight.gif +3 -0
README.md
CHANGED
@@ -181,13 +181,27 @@ CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/instance_level
|
|
181 |
```
|
182 |
|
183 |
<table class="center">
|
|
|
|
|
|
|
|
|
184 |
<tr>
|
185 |
<td><img src="assets/instance-level/left_iron_right_spider.gif"></td>
|
186 |
<td><img src="assets/instance-level/teddy_koala.gif"></td>
|
|
|
|
|
|
|
|
|
|
|
|
|
187 |
<td><img src="assets/instance-level/badminton.gif"></td>
|
|
|
|
|
|
|
|
|
|
|
188 |
</tr>
|
189 |
<tr>
|
190 |
-
<td><img src="assets/instance-level/bike_3.gif"></td>
|
191 |
<td><img src="assets/instance-level/panda_vs_poddle.gif"></td>
|
192 |
<td><img src="assets/instance-level/2cars.gif"></td>
|
193 |
</tr>
|
@@ -238,42 +252,47 @@ CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/part_level/modi
|
|
238 |
## Soely Edit on specific subjects, keep background unchanged
|
239 |
You can get soely video editing results, using the following command:
|
240 |
```bash
|
241 |
-
CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/
|
|
|
|
|
242 |
```
|
243 |
|
244 |
<table class="center">
|
245 |
<tr>
|
246 |
-
<td><img src="assets/
|
247 |
-
<td><img src="assets/
|
248 |
-
<td><img src="assets/
|
249 |
-
<td><img src="assets/
|
250 |
</tr>
|
251 |
<tr>
|
252 |
-
<td width=
|
253 |
-
<td width=
|
254 |
-
<td width=
|
255 |
-
<td width=
|
256 |
</tr>
|
257 |
</table>
|
258 |
|
259 |
-
##
|
260 |
You can get visulize attention weight editing results, using the following command:
|
261 |
```bash
|
262 |
-
|
|
|
263 |
```
|
264 |
|
265 |
<table class="center">
|
266 |
<tr>
|
267 |
-
<td><img src="assets/
|
268 |
-
<td><img src="assets/
|
269 |
-
<td><img src="assets/
|
270 |
-
<td><img src="assets/
|
|
|
271 |
</tr>
|
272 |
<tr>
|
273 |
-
<td width=
|
274 |
-
<td width=
|
275 |
-
<td width=
|
276 |
-
<td width=
|
|
|
277 |
</tr>
|
278 |
</table>
|
279 |
|
|
|
181 |
```
|
182 |
|
183 |
<table class="center">
|
184 |
+
<tr>
|
185 |
+
<td width=50% style="text-align:center;">running_two_man/3cls_iron_spider.yaml</td>
|
186 |
+
<td width=50% style="text-align:center;">2_monkeys/2cls_teddy_bear_koala.yaml</td>
|
187 |
+
</tr>
|
188 |
<tr>
|
189 |
<td><img src="assets/instance-level/left_iron_right_spider.gif"></td>
|
190 |
<td><img src="assets/instance-level/teddy_koala.gif"></td>
|
191 |
+
</tr>
|
192 |
+
<tr>
|
193 |
+
<td width=50% style="text-align:center;">badminton/2cls_wonder_woman_spiderman.yaml</td>
|
194 |
+
<td width=50% style="text-align:center;">soap-box/soap-box.yaml</td>
|
195 |
+
</tr>
|
196 |
+
<tr>
|
197 |
<td><img src="assets/instance-level/badminton.gif"></td>
|
198 |
+
<td><img src="assets/teaser/soap-box.gif"></td>
|
199 |
+
</tr>
|
200 |
+
<tr>
|
201 |
+
<td width=50% style="text-align:center;">2_cats/4cls_panda_vs_poddle.yaml</td>
|
202 |
+
<td width=50% style="text-align:center;">2_cars/left_firetruck_right_bus.yaml</td>
|
203 |
</tr>
|
204 |
<tr>
|
|
|
205 |
<td><img src="assets/instance-level/panda_vs_poddle.gif"></td>
|
206 |
<td><img src="assets/instance-level/2cars.gif"></td>
|
207 |
</tr>
|
|
|
252 |
## Soely Edit on specific subjects, keep background unchanged
|
253 |
You can get soely video editing results, using the following command:
|
254 |
```bash
|
255 |
+
CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/instance_level/soely_edit/only_left.yaml
|
256 |
+
#--config config/instance_level/soely_edit/only_right.yaml
|
257 |
+
#--config config/instance_level/soely_edit/joint_edit.yaml
|
258 |
```
|
259 |
|
260 |
<table class="center">
|
261 |
<tr>
|
262 |
+
<td><img src="assets/soely_edit/input.gif"></td>
|
263 |
+
<td><img src="assets/soely_edit/left.gif"></td>
|
264 |
+
<td><img src="assets/soely_edit/right.gif"></td>
|
265 |
+
<td><img src="assets/soely_edit/joint.gif"></td>
|
266 |
</tr>
|
267 |
<tr>
|
268 |
+
<td width=25% style="text-align:center;">source video</td>
|
269 |
+
<td width=25% style="text-align:center;">left→Iron Man</td>
|
270 |
+
<td width=25% style="text-align:center;">right→Spiderman</td>
|
271 |
+
<td width=25% style="text-align:center;">joint edit</td>
|
272 |
</tr>
|
273 |
</table>
|
274 |
|
275 |
+
## 放大镜 Visualize Cross Attention Weight
|
276 |
You can get visulize attention weight editing results, using the following command:
|
277 |
```bash
|
278 |
+
#setting vis_cross_attn: True in your config
|
279 |
+
CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/instance_level/running_two_man/3cls_spider_polar_vis_weight.yaml
|
280 |
```
|
281 |
|
282 |
<table class="center">
|
283 |
<tr>
|
284 |
+
<td><img src="assets/soely_edit/input.gif"></td>
|
285 |
+
<td><img src="assets/soely_edit/edit.gif"></td>
|
286 |
+
<td><img src="assets/vis/spiderman_weight.gif"></td>
|
287 |
+
<td><img src="assets/vis/bear_weight.gif"></td>
|
288 |
+
<td><img src="/assets/vis/cherry_weight.gif"></td>
|
289 |
</tr>
|
290 |
<tr>
|
291 |
+
<td width=20% style="text-align:center;">source video</td>
|
292 |
+
<td width=20% style="text-align:center;">left→spiderman, right→polar bear, trees→cherry blossoms</td>
|
293 |
+
<td width=20% style="text-align:center;">spiderman weight</td>
|
294 |
+
<td width=20% style="text-align:center;">bear weight</td>
|
295 |
+
<td width=20% style="text-align:center;">cherry weight</td>
|
296 |
</tr>
|
297 |
</table>
|
298 |
|
assets/soely_edit/input.gif
ADDED
![]() |
Git LFS Details
|
assets/soely_edit/joint.gif
ADDED
![]() |
Git LFS Details
|
assets/soely_edit/left.gif
ADDED
![]() |
Git LFS Details
|
assets/soely_edit/right.gif
ADDED
![]() |
Git LFS Details
|
assets/vis/bear_weight.gif
ADDED
![]() |
Git LFS Details
|
assets/vis/cherry_weight.gif
ADDED
![]() |
Git LFS Details
|
assets/vis/edit.gif
ADDED
![]() |
Git LFS Details
|
assets/vis/spiderman_weight.gif
ADDED
![]() |
Git LFS Details
|