| # Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy | |
| [[Homepage](https://nl2hltl.github.io/)] [[Paper](https://arxiv.org/abs/2408.08188)] [[Video](https://youtu.be/o8CRrVK9g9Q)] [[Poster](https://nl2hltl.github.io/media/figures/overview.png)] | |
| > The associated repo for paper "Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy". | |
| ## Introduction | |
| The dataset is modified based on the following projects, in which we replace the task related descriptions, into task independent descriptions | |
| task related NL2TL example (from [Efficient-Eng-2-LTL](https://github.com/UM-ARM-Lab/Efficient-Eng-2-LTL)) | |
| ```json | |
| "globally ( and ( until ( scan , not ( any cubes ) ) , finally ( any cubes ) ) )": { | |
| "formula": "globally ( and ( until ( scan , not ( any cubes ) ) , finally ( any cubes ) ) )", | |
| "raw": "G & U S ! A F A" | |
| }, | |
| ``` | |
| task independent NL2TL example: | |
| ```json | |
| {"natural": "go through the P01 until you get to the P04", "raw_ltl": "F ( P01 A ( F P04 ) )"} | |
| ``` | |
| **NOTE:** We mechanically obtain task independent descriptions from task related descriptions by noun/phrase substitution. Due to the removal of semantic information, some NL2TL mappings obtained through this method are not unique. | |
| Based task related NL2TL datasets: | |
| - datasets | |
| - [Efficient-Eng-2-LTL](https://github.com/UM-ARM-Lab/Efficient-Eng-2-LTL) | |
| - [Lang2LTL](https://github.com/h2r/Lang2LTL) | |
| - [nl2spec](https://github.com/realChrisHahn2/nl2spec) | |
| - [NL2TL](https://github.com/yongchao98/NL2TL) | |
| ## File Structure | |
| - NL2HLTLTranslator | |
| - fastapi_server.py a FastAPI server for translate testing, will run on localhost:8001 | |
| - mistral7b | |
| - finetune.py code for fintune | |
| - prediction.py code for prediction (this version do not have sockets) | |
| - mistral7b_quat8: a fintuned model based on Mistral7B in quat 8 | |
| - NL2TL-dataset: used dataset | |
| ## Run | |
| ```bash | |
| cd to/this/folder | |
| pip install -e . | |
| python finetune/fastapi_server.py | |
| ``` | |
| ## Cite | |
| ```bibtex | |
| @misc{xu2024scalingnaturallanguageunderstanding, | |
| title={Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy}, | |
| author={Shaojun Xu and Xusheng Luo and Yutong Huang and Letian Leng and Ruixuan Liu and Changliu Liu}, | |
| year={2024}, | |
| eprint={2408.08188}, | |
| archivePrefix={arXiv}, | |
| primaryClass={cs.RO}, | |
| url={https://arxiv.org/abs/2408.08188}, | |
| } | |
| ``` | |