|
# Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy
|
|
|
|
[[Homepage](https://nl2hltl.github.io/)] [[Paper](https://arxiv.org/abs/2408.08188)] [[Video](https://youtu.be/o8CRrVK9g9Q)] [[Poster](https://nl2hltl.github.io/media/figures/overview.png)]
|
|
|
|
> The associated repo for paper "Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy".
|
|
|
|
## Introduction
|
|
|
|
The dataset is modified based on the following projects, in which we replace the task related descriptions, into task independent descriptions
|
|
|
|
task related NL2TL example (from [Efficient-Eng-2-LTL](https://github.com/UM-ARM-Lab/Efficient-Eng-2-LTL))
|
|
|
|
```json
|
|
"globally ( and ( until ( scan , not ( any cubes ) ) , finally ( any cubes ) ) )": {
|
|
"formula": "globally ( and ( until ( scan , not ( any cubes ) ) , finally ( any cubes ) ) )",
|
|
"raw": "G & U S ! A F A"
|
|
},
|
|
```
|
|
|
|
task independent NL2TL example:
|
|
```json
|
|
{"natural": "go through the P01 until you get to the P04", "raw_ltl": "F ( P01 A ( F P04 ) )"}
|
|
```
|
|
|
|
**NOTE:** We mechanically obtain task independent descriptions from task related descriptions by noun/phrase substitution. Due to the removal of semantic information, some NL2TL mappings obtained through this method are not unique.
|
|
|
|
Based task related NL2TL datasets:
|
|
- datasets
|
|
- [Efficient-Eng-2-LTL](https://github.com/UM-ARM-Lab/Efficient-Eng-2-LTL)
|
|
- [Lang2LTL](https://github.com/h2r/Lang2LTL)
|
|
- [nl2spec](https://github.com/realChrisHahn2/nl2spec)
|
|
- [NL2TL](https://github.com/yongchao98/NL2TL)
|
|
## File Structure
|
|
- NL2HLTLTranslator
|
|
- fastapi_server.py a FastAPI server for translate testing, will run on localhost:8001
|
|
- mistral7b
|
|
- finetune.py code for fintune
|
|
- prediction.py code for prediction (this version do not have sockets)
|
|
- mistral7b_quat8: a fintuned model based on Mistral7B in quat 8
|
|
- NL2TL-dataset: used dataset
|
|
## Run
|
|
```bash
|
|
cd to/this/folder
|
|
pip install -e .
|
|
python finetune/fastapi_server.py
|
|
```
|
|
## Cite
|
|
```bibtex
|
|
@misc{xu2024scalingnaturallanguageunderstanding,
|
|
title={Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy},
|
|
author={Shaojun Xu and Xusheng Luo and Yutong Huang and Letian Leng and Ruixuan Liu and Changliu Liu},
|
|
year={2024},
|
|
eprint={2408.08188},
|
|
archivePrefix={arXiv},
|
|
primaryClass={cs.RO},
|
|
url={https://arxiv.org/abs/2408.08188},
|
|
}
|
|
```
|
|
|