XiangpengYang commited on
Commit
5357c60
·
1 Parent(s): dd95d8d
README.md CHANGED
@@ -181,13 +181,27 @@ CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/instance_level
181
  ```
182
 
183
  <table class="center">
 
 
 
 
184
  <tr>
185
  <td><img src="assets/instance-level/left_iron_right_spider.gif"></td>
186
  <td><img src="assets/instance-level/teddy_koala.gif"></td>
 
 
 
 
 
 
187
  <td><img src="assets/instance-level/badminton.gif"></td>
 
 
 
 
 
188
  </tr>
189
  <tr>
190
- <td><img src="assets/instance-level/bike_3.gif"></td>
191
  <td><img src="assets/instance-level/panda_vs_poddle.gif"></td>
192
  <td><img src="assets/instance-level/2cars.gif"></td>
193
  </tr>
@@ -238,42 +252,47 @@ CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/part_level/modi
238
  ## Soely Edit on specific subjects, keep background unchanged
239
  You can get soely video editing results, using the following command:
240
  ```bash
241
- CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/part_level/modification/man_text_message/blue_shirt.yaml
 
 
242
  ```
243
 
244
  <table class="center">
245
  <tr>
246
- <td><img src="assets/part-level/man_text_message.gif"></td>
247
- <td><img src="assets/part-level/blue-shirt.gif"></td>
248
- <td><img src="assets/part-level/black-suit.gif"></td>
249
- <td><img src="assets/part-level/cat_flower.gif"></td>
250
  </tr>
251
  <tr>
252
- <td width=15% style="text-align:center;">source video</td>
253
- <td width=15% style="text-align:center;">blue shirt</td>
254
- <td width=15% style="text-align:center;">black suit</td>
255
- <td width=15% style="text-align:center;">source video</td>
256
  </tr>
257
  </table>
258
 
259
- ## 🔍 Visualize Cross Attention Weight
260
  You can get visulize attention weight editing results, using the following command:
261
  ```bash
262
- CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/part_level/modification/man_text_message/blue_shirt.yaml
 
263
  ```
264
 
265
  <table class="center">
266
  <tr>
267
- <td><img src="assets/part-level/man_text_message.gif"></td>
268
- <td><img src="assets/part-level/blue-shirt.gif"></td>
269
- <td><img src="assets/part-level/black-suit.gif"></td>
270
- <td><img src="assets/part-level/cat_flower.gif"></td>
 
271
  </tr>
272
  <tr>
273
- <td width=15% style="text-align:center;">source video</td>
274
- <td width=15% style="text-align:center;">blue shirt</td>
275
- <td width=15% style="text-align:center;">black suit</td>
276
- <td width=15% style="text-align:center;">source video</td>
 
277
  </tr>
278
  </table>
279
 
 
181
  ```
182
 
183
  <table class="center">
184
+ <tr>
185
+ <td width=50% style="text-align:center;">running_two_man/3cls_iron_spider.yaml</td>
186
+ <td width=50% style="text-align:center;">2_monkeys/2cls_teddy_bear_koala.yaml</td>
187
+ </tr>
188
  <tr>
189
  <td><img src="assets/instance-level/left_iron_right_spider.gif"></td>
190
  <td><img src="assets/instance-level/teddy_koala.gif"></td>
191
+ </tr>
192
+ <tr>
193
+ <td width=50% style="text-align:center;">badminton/2cls_wonder_woman_spiderman.yaml</td>
194
+ <td width=50% style="text-align:center;">soap-box/soap-box.yaml</td>
195
+ </tr>
196
+ <tr>
197
  <td><img src="assets/instance-level/badminton.gif"></td>
198
+ <td><img src="assets/teaser/soap-box.gif"></td>
199
+ </tr>
200
+ <tr>
201
+ <td width=50% style="text-align:center;">2_cats/4cls_panda_vs_poddle.yaml</td>
202
+ <td width=50% style="text-align:center;">2_cars/left_firetruck_right_bus.yaml</td>
203
  </tr>
204
  <tr>
 
205
  <td><img src="assets/instance-level/panda_vs_poddle.gif"></td>
206
  <td><img src="assets/instance-level/2cars.gif"></td>
207
  </tr>
 
252
  ## Soely Edit on specific subjects, keep background unchanged
253
  You can get soely video editing results, using the following command:
254
  ```bash
255
+ CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/instance_level/soely_edit/only_left.yaml
256
+ #--config config/instance_level/soely_edit/only_right.yaml
257
+ #--config config/instance_level/soely_edit/joint_edit.yaml
258
  ```
259
 
260
  <table class="center">
261
  <tr>
262
+ <td><img src="assets/soely_edit/input.gif"></td>
263
+ <td><img src="assets/soely_edit/left.gif"></td>
264
+ <td><img src="assets/soely_edit/right.gif"></td>
265
+ <td><img src="assets/soely_edit/joint.gif"></td>
266
  </tr>
267
  <tr>
268
+ <td width=25% style="text-align:center;">source video</td>
269
+ <td width=25% style="text-align:center;">left→Iron Man</td>
270
+ <td width=25% style="text-align:center;">right→Spiderman</td>
271
+ <td width=25% style="text-align:center;">joint edit</td>
272
  </tr>
273
  </table>
274
 
275
+ ## 放大镜 Visualize Cross Attention Weight
276
  You can get visulize attention weight editing results, using the following command:
277
  ```bash
278
+ #setting vis_cross_attn: True in your config
279
+ CUDA_VISIBLE_DEVICES=0 accelerate launch test.py --config config/instance_level/running_two_man/3cls_spider_polar_vis_weight.yaml
280
  ```
281
 
282
  <table class="center">
283
  <tr>
284
+ <td><img src="assets/soely_edit/input.gif"></td>
285
+ <td><img src="assets/soely_edit/edit.gif"></td>
286
+ <td><img src="assets/vis/spiderman_weight.gif"></td>
287
+ <td><img src="assets/vis/bear_weight.gif"></td>
288
+ <td><img src="/assets/vis/cherry_weight.gif"></td>
289
  </tr>
290
  <tr>
291
+ <td width=20% style="text-align:center;">source video</td>
292
+ <td width=20% style="text-align:center;">left→spiderman, right→polar bear, trees→cherry blossoms</td>
293
+ <td width=20% style="text-align:center;">spiderman weight</td>
294
+ <td width=20% style="text-align:center;">bear weight</td>
295
+ <td width=20% style="text-align:center;">cherry weight</td>
296
  </tr>
297
  </table>
298
 
assets/soely_edit/input.gif ADDED

Git LFS Details

  • SHA256: ed0dff7d0d5c58fcc489ef32edfff3307b08d62856e4e2646b1fdfbeb7f3205b
  • Pointer size: 132 Bytes
  • Size of remote file: 2.47 MB
assets/soely_edit/joint.gif ADDED

Git LFS Details

  • SHA256: b53377bd4209578d1e5ea4dbf1ec0735ac6111e1fa2480255c712b0ea02f4c18
  • Pointer size: 132 Bytes
  • Size of remote file: 2.64 MB
assets/soely_edit/left.gif ADDED

Git LFS Details

  • SHA256: 82c959ca3f3331a50c0a325f71a7fc56dd6b80ef95543928b33c1ca3261b1f14
  • Pointer size: 132 Bytes
  • Size of remote file: 2.62 MB
assets/soely_edit/right.gif ADDED

Git LFS Details

  • SHA256: 0826a0abca4daef4e618b9ac7964593df1dfe840313741b8774fc8bf441c985d
  • Pointer size: 132 Bytes
  • Size of remote file: 2.33 MB
assets/vis/bear_weight.gif ADDED

Git LFS Details

  • SHA256: e6eaad7d1bfe98e227e05688fea046498295efba61f0e8806935f324a0db1659
  • Pointer size: 131 Bytes
  • Size of remote file: 729 kB
assets/vis/cherry_weight.gif ADDED

Git LFS Details

  • SHA256: f1c7df23d2a55eeb7fffbaf65191d00e7a014abf49ff3f98f4ba4577eb4e1731
  • Pointer size: 131 Bytes
  • Size of remote file: 728 kB
assets/vis/edit.gif ADDED

Git LFS Details

  • SHA256: c384bb3c35e8553a6987f116d690677f29997070e591c3f1aa443db43b7ca1e5
  • Pointer size: 132 Bytes
  • Size of remote file: 2.43 MB
assets/vis/spiderman_weight.gif ADDED

Git LFS Details

  • SHA256: 2c81f6a36f5f125b15342f64f9684173172da1f1fb5dc78f69a752d1974f3925
  • Pointer size: 131 Bytes
  • Size of remote file: 694 kB