rubentito commited on
Commit
b38f8b9
1 Parent(s): db7728d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -51,6 +51,14 @@ answer_tokens = context_tokens[start_pos: end_pos]
51
  answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
52
  ```
53
 
 
 
 
 
 
 
 
 
54
  ## Model results
55
 
56
  Extended experimentation can be found in Table 2 of [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/pdf/2212.05935.pdf).
 
51
  answer = tokenizer.decode(tokenizer.convert_tokens_to_ids(answer_tokens))
52
  ```
53
 
54
+ ## Metrics
55
+ ### Average Normalized Levenshtein Similarity (ANLS):
56
+
57
+ The standard metric for text-based VQA tasks (ST-VQA and DocVQA). It evaluates the method's reasoning capabilities while smoothly penalizes OCR recognition errors. For more information check [Scene Text Visual Question Answering](https://arxiv.org/abs/1905.13648)
58
+
59
+ ### Answer Page Prediction Accuracy (APPA):
60
+ In the MP-DocVQA task, the models can provide the index of the page where the information required to answer the question is located. For this subtask accuracy is used to evaluate the predictions: i.e. if the predicted page is correct or not. For more information, check [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/abs/2212.05935)
61
+
62
  ## Model results
63
 
64
  Extended experimentation can be found in Table 2 of [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/pdf/2212.05935.pdf).