Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy

[Homepage] [Paper] [Video] [Poster]

The associated repo for paper "Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy".

Introduction

The dataset is modified based on the following projects, in which we replace the task related descriptions, into task independent descriptions

task related NL2TL example (from Efficient-Eng-2-LTL)

"globally ( and ( until ( scan , not ( any cubes ) ) , finally ( any cubes ) ) )": {
    "formula": "globally ( and ( until ( scan , not ( any cubes ) ) , finally ( any cubes ) ) )",
    "raw": "G & U S ! A F A"
  },

task independent NL2TL example:

{"natural": "go through the P01 until you get to the  P04", "raw_ltl": "F ( P01 A ( F P04 ) )"}

NOTE: We mechanically obtain task independent descriptions from task related descriptions by noun/phrase substitution. Due to the removal of semantic information, some NL2TL mappings obtained through this method are not unique.

Based task related NL2TL datasets:

datasets

File Structure

NL2HLTLTranslator
- fastapi_server.py a FastAPI server for translate testing, will run on localhost:8001
- mistral7b
  - finetune.py code for fintune
  - prediction.py code for prediction (this version do not have sockets)
mistral7b_quat8: a fintuned model based on Mistral7B in quat 8
NL2TL-dataset: used dataset

Run

cd to/this/folder
pip install -e .
python finetune/fastapi_server.py

Cite

@misc{xu2024scalingnaturallanguageunderstanding,
      title={Scaling Up Natural Language Understanding for Multi-Robots Through the Lens of Hierarchy}, 
      author={Shaojun Xu and Xusheng Luo and Yutong Huang and Letian Leng and Ruixuan Liu and Changliu Liu},
      year={2024},
      eprint={2408.08188},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2408.08188}, 
}