youngzhou12 commited on
Commit
fa2c3bc
·
verified ·
1 Parent(s): 19e4196

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +87 -3
README.md CHANGED
@@ -1,3 +1,87 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: pytorch
4
+ tags:
5
+ - Medical Vsion-Language Pre-Training
6
+ - BenchX
7
+ ---
8
+
9
+ # M-FLAG Checkpoint Model Card
10
+
11
+ A retrained M-FLAG model for benchmarking medical vision-language pre-training methods within the BenchX framework.
12
+
13
+ ## Model Details
14
+ - **Model Type**: M-FLAG
15
+ - **Architecture**: ResNet-50 image encoder and CXR-BERT text encoder
16
+ - **Original Papers**: [M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization](https://arxiv.org/abs/2307.08347)
17
+ - **Benchmark Paper**: [BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rays](https://arxiv.org/abs/2410.21969)
18
+ - **Benchmark Framework**: https://github.com/yangzhou12/BenchX
19
+
20
+ ## Intended Use
21
+ - **Primary Use Cases**:
22
+ - Benchmarking performance for Medical Image Classification
23
+ - Benchmarking performance for Medical Image Segmentation
24
+ - Benchmarking performance for Medical Report Generation
25
+
26
+ ## Pre-Training Data
27
+ - **Dataset**:
28
+ - Data source(s): MIMIC-CXR
29
+ - Types of medical images: Frontal chest X-rays
30
+ - Text data type: Associated radiology reports
31
+
32
+ ## Prerequisites
33
+
34
+ Please follow the [instruction](https://github.com/yangzhou12/BenchX/blob/release/README.md#installation) to install BenchX.
35
+
36
+ ## Training & Evaluation
37
+
38
+ ### 1. Classification
39
+
40
+ To fine-tune M-FLAG for classification, run this command:
41
+
42
+ ```
43
+ python bin/train.py config/classification/<dataset_name>/M-FLAG.yml
44
+ ```
45
+
46
+ ### 2. Segmentation
47
+ To fine-tune M-FLAG for segmentation, run this command:
48
+
49
+ ```
50
+ python mmsegmentation/tools/train.py config/benchmark/<dataset_name>/M-FLAG.yml
51
+ ```
52
+
53
+ ### 3. Report Generation
54
+ To fine-tune M-FLAG for report generation, run this command:
55
+ ```
56
+ python bin/train.py config/report_generation/<dataset_name>/M-FLAG.yml
57
+ ```
58
+
59
+ ### 4. Evaluation
60
+ To evaluate fine-tuned M-FLAG models, run:
61
+
62
+ ```
63
+ # For classification and report generation
64
+ python bin/test.py config/<task_name>/<dataset_name>/M-FLAG.yml validator.splits=[test] ckpt_dir=<path_to_checkpoint>
65
+
66
+ # For segmentation
67
+ python mmsegmentation/tools/my_test.py mmsegmentation/config/<dataset_name>/M-FLAG.yml <path_to_checkpoint>
68
+ ```
69
+
70
+ ## Citations
71
+ ```bibtex
72
+ @inproceedings{huang2021M-FLAG,
73
+ title={M-FLAG: Medical Vision-Language Pre-training with Frozen Language Models and Latent Space Geometry Optimization},
74
+ author={Liu, Che and Cheng, Sibo and Chen, Chen and Qiao, Mengyun and Zhang, Weitong and Shah, Anand and Bai, Wenjia and Arcucci, Rossella},
75
+ booktitle={Proceedings of MICCAI},
76
+ pages={637--647},
77
+ year={2023},
78
+ }
79
+ ```
80
+ ```bibtex
81
+ @inproceedings{zhou2024benchx,
82
+ title={BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rays},
83
+ author={Yang Zhou, Tan Li Hui Faith, Yanyu Xu, Sicong Leng, Xinxing Xu, Yong Liu, Rick Siow Mong Goh},
84
+ booktitle={Proceedings of NeurIPS},
85
+ year={2024}
86
+ }
87
+ ```