yhsieh commited on
Commit
23f9bed
·
verified ·
1 Parent(s): 36c1525

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apple-ascl
3
+ datasets:
4
+ - graph-based-captions/GBC10M
5
+ language:
6
+ - en
7
+ ---
8
+
9
+ ### Graph-based captioning (GBC) is a new image annotation paradigm that combines the strengths of long captions, region captions, and scene graphs
10
+
11
+ GBC interconnects region captions to create a unified description akin to a long caption, while also providing structural information similar to scene graphs.
12
+ ![assets/GBC_illustration.png](assets/GBC_illustration.png)
13
+
14
+
15
+ ### Text-to-Image with GBC as Middleware
16
+
17
+ We propose to use GBC as middleware for text-to-image generation. This repository provides model for generating GBC annotation from simple text prompt.
18
+ ![assets/GBC_promptgen.png](assets/GBC_promptgen.png)
19
+
20
+ For futher detail on how to use the model please refer to the [accompanying code repository](https://github.com/apple/ml-gbc?tab=readme-ov-file#-gbc-text-to-image).
21
+
22
+ ### License
23
+
24
+ For license please checkout the [LICENSE](LICENSE) file.
25
+
26
+ ### Citation
27
+
28
+ ```
29
+ @article{GBC2024,
30
+ title={Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions},
31
+ author={Yu-Guan Hsieh and Cheng-Yu Hsieh and Shih-Ying Yeh and Louis Béthune and Hadi Pouransari and Pavan Kumar Anasosalu Vasu and Chun-Liang Li and Ranjay Krishna and Oncel Tuzel and Marco Cuturi},
32
+ journal={arXiv preprint arXiv:2407.06723},
33
+ year={2024}
34
+ }