Update README.md
Browse files
README.md
CHANGED
@@ -1,15 +1,34 @@
|
|
1 |
-
---
|
2 |
-
license: apache-2.0
|
3 |
-
datasets:
|
4 |
-
- declare-lab/Emma-X-GCOT
|
5 |
-
metrics:
|
6 |
-
- accuracy
|
7 |
-
base_model:
|
8 |
-
- openvla/openvla-7b
|
9 |
-
|
10 |
-
|
11 |
-
|
12 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
13 |
|
14 |
EMMA-X is an Embodied Multimodal Action (VLA) Model designed to bridge the gap between Visual-Language Models (VLMs) and robotic control tasks. EMMA-X generalizes effectively across diverse environments, objects, and instructions while excelling at long-horizon spatial reasoning and grounded task planning using a novel Trajectory Segmentation Strategy.
|
15 |
|
@@ -68,4 +87,14 @@ action, grounded_reasoning = vla.generate_actions(
|
|
68 |
print("Grounded Reasoning:", grounded_reasoning)
|
69 |
# Execute...
|
70 |
robot.act(action, ...)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
71 |
```
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
datasets:
|
4 |
+
- declare-lab/Emma-X-GCOT
|
5 |
+
metrics:
|
6 |
+
- accuracy
|
7 |
+
base_model:
|
8 |
+
- openvla/openvla-7b
|
9 |
+
pipeline_tag: image-text-to-text
|
10 |
+
---
|
11 |
+
|
12 |
+
<h1 align="center">✨
|
13 |
+
<br/>
|
14 |
+
Meet Emma-X, an Embodied Multimodal Action Model
|
15 |
+
<br/>
|
16 |
+
✨✨✨
|
17 |
+
|
18 |
+
|
19 |
+
</h1>
|
20 |
+
|
21 |
+
<div align="center">
|
22 |
+
<img src="https://raw.githubusercontent.com/declare-lab/Emma-X/main/Emma-X.png" alt="Emma-X" width="300" />
|
23 |
+
|
24 |
+
<br/>
|
25 |
+
|
26 |
+
[](https://arxiv.org/abs/2412.11974) [](https://huggingface.co/declare-lab/Emma-X) [](https://declare-lab.github.io/Emma-X/)
|
27 |
+
|
28 |
+
|
29 |
+
</div>
|
30 |
+
|
31 |
+
## Model Overview
|
32 |
|
33 |
EMMA-X is an Embodied Multimodal Action (VLA) Model designed to bridge the gap between Visual-Language Models (VLMs) and robotic control tasks. EMMA-X generalizes effectively across diverse environments, objects, and instructions while excelling at long-horizon spatial reasoning and grounded task planning using a novel Trajectory Segmentation Strategy.
|
34 |
|
|
|
87 |
print("Grounded Reasoning:", grounded_reasoning)
|
88 |
# Execute...
|
89 |
robot.act(action, ...)
|
90 |
+
```
|
91 |
+
|
92 |
+
## Citation
|
93 |
+
```
|
94 |
+
@article{sun2024emma,
|
95 |
+
title={Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoning},
|
96 |
+
author={Sun, Qi and Hong, Pengfei and Pala, Tej Deep and Toh, Vernon and Tan, U-Xuan and Ghosal, Deepanway and Poria, Soujanya},
|
97 |
+
journal={arXiv preprint arXiv:2412.11974},
|
98 |
+
year={2024}
|
99 |
+
}
|
100 |
```
|