Update LMFlow support (#6)

- Update LMFlow support (88cd2a92ef203e8eee99b981498f9593fa99f97f)


Co-authored-by: Shizhe <[email protected]>

README.md

@@ -115,6 +115,41 @@ print(f"Model response: {response}")
 
 ```
 
+## Finetuning Hymba
+
+
+[LMFlow](https://github.com/OptimalScale/LMFlow) is a complete pipeline for fine-tuning large language models. 
+The following steps provide an example of how to fine-tune the `Hymba-1.5B-Base` models using LMFlow.
+
+1. Using Docker
+
+    ```
+      docker pull ghcr.io/tilmto/hymba:v1
+      docker run --gpus all -v /home/$USER:/home/$USER -it ghcr.io/tilmto/hymba:v1 bash
+    ```
+2. Install LMFlow
+
+    ```
+      git clone https://github.com/OptimalScale/LMFlow.git
+      cd LMFlow
+      conda create -n lmflow python=3.9 -y
+      conda activate lmflow
+      conda install mpi4py
+      pip install -e .
+    ```
+
+3. Fine-tune the model using the following command.
+  
+    ```
+      cd LMFlow
+      bash ./scripts/run_finetune_hymba.sh
+    ```
+
+With LMFlow, you can also fine-tune the model on your custom dataset. The only thing you need to do is transform your dataset into the [LMFlow data format](https://optimalscale.github.io/LMFlow/examples/DATASETS.html).
+In addition to full-finetuniing, you can also fine-tune hymba efficiently with [DoRA](https://arxiv.org/html/2402.09353v4), [LoRA](https://github.com/OptimalScale/LMFlow?tab=readme-ov-file#lora), [LISA](https://github.com/OptimalScale/LMFlow?tab=readme-ov-file#lisa), [Flash Attention](https://github.com/OptimalScale/LMFlow/blob/main/readme/flash_attn2.md), and other acceleration techniques.
+For more details, please refer to the [LMFlow for Hymba](https://github.com/OptimalScale/LMFlow/tree/main/experimental/Hymba) documentation.
+
+
 ## Evaluation 
 We use [`LM Evaluation Harness`](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the model. The evaluation commands are as follows: