liujie commited on
Commit
a6a6e7c
·
1 Parent(s): c744934

first commit

Browse files
Files changed (1) hide show
  1. src/md.py +1 -0
src/md.py CHANGED
@@ -7,6 +7,7 @@ ABOUT_TEXT = """
7
  The prompt-video pairs are sourced from [VideoGen-Eval](https://ailab-cvc.github.io/VideoGen-Eval/), a dataset featuring a diverse range of prompts and videos generated by state-of-the-art video diffusion models (VDMs). Our benchmark comprises 26.5k video pairs, each annotated with a corresponding preference label.
8
  <img src="https://i.postimg.cc/J7XhVLTh/image.png" alt="Video Duration and Resolution in VideoGen-RewardBench" style="width: 400px;"/>
9
 
 
10
  We report two accuracy metrics: ties-included accuracy **(w/ Ties)** and ties-excluded accuracy **(w/o Ties)**.
11
 
12
  - For ties-excluded accuracy, we exclude all data labeled as ”ties” and use only data labeled as ”A wins” or ”B wins” for calculation. We compute the rewards for each prompt-video pair, convert the relative reward relationships into binary labels, and calculate classification accuracy.
 
7
  The prompt-video pairs are sourced from [VideoGen-Eval](https://ailab-cvc.github.io/VideoGen-Eval/), a dataset featuring a diverse range of prompts and videos generated by state-of-the-art video diffusion models (VDMs). Our benchmark comprises 26.5k video pairs, each annotated with a corresponding preference label.
8
  <img src="https://i.postimg.cc/J7XhVLTh/image.png" alt="Video Duration and Resolution in VideoGen-RewardBench" style="width: 400px;"/>
9
 
10
+
11
  We report two accuracy metrics: ties-included accuracy **(w/ Ties)** and ties-excluded accuracy **(w/o Ties)**.
12
 
13
  - For ties-excluded accuracy, we exclude all data labeled as ”ties” and use only data labeled as ”A wins” or ”B wins” for calculation. We compute the rewards for each prompt-video pair, convert the relative reward relationships into binary labels, and calculate classification accuracy.