Update README.md

README.md

@@ -1,16 +1,16 @@
 ---
 license: apache-2.0
 ---
-### Huggingface RWKV Finch 7B Model
+### Huggingface RWKV Flock of Finches 36B-A11B Mixture of Experts Model
 
-> HF compatible model for Finch-7B.
+> HF compatible model for Finch-MoE-36B-A11B.
 
 ![Finch Bird](./imgs/finch.jpg)
 
 
 > **! Important Note !**
 >
-> The following is the HF transformers implementation of the Finch 7B model. This is meant to be used with the huggingface transformers
+> The following is the HF transformers implementation of the Flock of Finches Mixture of Experts 36B-A11B model. This is meant to be used with the huggingface transformers
 >
 >
 
@@ -18,21 +18,21 @@ license: apache-2.0
 ## Quickstart with the hugging face transformer library
 
 ```
-model = AutoModelForCausalLM.from_pretrained("RWKV/v6-Finch-7B-HF", trust_remote_code=True).to(torch.float32)
-tokenizer = AutoTokenizer.from_pretrained("RWKV/v6-Finch-7B-HF", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained("RWKV/Finch-MoE-36B-A11B-HF", trust_remote_code=True).to(torch.float32)
+tokenizer = AutoTokenizer.from_pretrained("RWKV/Finch-MoE-36B-A11B-HF", trust_remote_code=True)
 ```
 
 ## Evaluation
 
-The following demonstrates the improvements from Eagle 7B to Finch 14B
+The following demonstrates the improvements from Eagle 7B to Flock of Finches 36B-A11B
 
-|  | [Eagle 7B](https://huggingface.co/RWKV/v6-Finch-7B-HF) | [Finch 7B](https://huggingface.co/RWKV/v6-Finch-7B-HF) | [Finch 14B](https://huggingface.co/RWKV/v6-Finch-14B-HF) |
-| --- | --- | --- | --- |
-| [ARC](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/arc) | 39.59% | 41.47% | 46.33% |
-| [HellaSwag](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/hellaswag) | 53.09% | 55.96% | 57.69% |
-| [MMLU](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/mmlu) | 30.86% | 41.70% | 56.05% |
-| [Truthful QA](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/truthfulqa) | 33.03% | 34.82% | 39.27% |
-| [Winogrande](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/winogrande) | 67.56% | 71.19% | 74.43% |
+|  | [Eagle 7B](https://huggingface.co/RWKV/v6-Finch-7B-HF) | [Finch 7B](https://huggingface.co/RWKV/v6-Finch-7B-HF) | [Finch 14B](https://huggingface.co/RWKV/v6-Finch-14B-HF) | [Flock of Finches 36B-A11B]
+| --- | --- | --- | --- | --- |
+| [ARC](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/arc) | 39.59% | 41.47% | 46.33% | 48.21%
+| [HellaSwag](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/hellaswag) | 53.09% | 55.96% | 57.69% | 57.69%
+| [MMLU](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/mmlu) | 30.86% | 41.70% | 56.05% | 55.0%
+| [Truthful QA](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/truthfulqa) | 33.03% | 34.82% | 39.27% | 
+| [Winogrande](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/winogrande) | 67.56% | 71.19% | 74.43% | 75.77%
 
 #### Running on CPU via HF transformers
 
@@ -59,8 +59,8 @@ User: {instruction}
 Assistant:"""
 
 
-model = AutoModelForCausalLM.from_pretrained("RWKV/v6-Finch-7B-HF", trust_remote_code=True).to(torch.float32)
-tokenizer = AutoTokenizer.from_pretrained("RWKV/v6-Finch-7B-HF", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained("RWKV/Finch-MoE-36B-A11B-HF", trust_remote_code=True).to(torch.float32)
+tokenizer = AutoTokenizer.from_pretrained("RWKV/Finch-MoE-36B-A11B-HF", trust_remote_code=True)
 
 text = "请介绍北京的旅游景点"
 prompt = generate_prompt(text)
@@ -115,8 +115,8 @@ User: {instruction}
 Assistant:"""
 
 
-model = AutoModelForCausalLM.from_pretrained("RWKV/v6-Finch-7B-HF", trust_remote_code=True, torch_dtype=torch.float16).to(0)
-tokenizer = AutoTokenizer.from_pretrained("RWKV/v6-Finch-7B-HF", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained("RWKV/Finch-MoE-36B-A11B-HF", trust_remote_code=True, torch_dtype=torch.float16).to(0)
+tokenizer = AutoTokenizer.from_pretrained("RWKV/Finch-MoE-36B-A11B-HF", trust_remote_code=True)
 
 text = "介绍一下大熊猫"
 prompt = generate_prompt(text)
@@ -162,8 +162,8 @@ User: {instruction}
 
 Assistant:"""
 
-model = AutoModelForCausalLM.from_pretrained("RWKV/v6-Finch-7B-HF", trust_remote_code=True).to(torch.float32)
-tokenizer = AutoTokenizer.from_pretrained("RWKV/v6-Finch-7B-HF", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained("RWKV/Finch-MoE-36B-A11B-HF", trust_remote_code=True).to(torch.float32)
+tokenizer = AutoTokenizer.from_pretrained("RWKV/Finch-MoE-36B-A11B-HF", trust_remote_code=True)
 
 texts = ["请介绍北京的旅游景点", "介绍一下大熊猫", "乌兰察布"]
 prompts = [generate_prompt(text) for text in texts]
@@ -208,8 +208,7 @@ Assistant: 乌兰察布是中国新疆维吾尔自治区的一个县级市，位
 ## Links
 - [Our wiki](https://wiki.rwkv.com)
 - [Recursal.AI Cloud Platform](https://recursal.ai)
-- [Featherless Inference](https://featherless.ai/models/RWKV/Finch-14B)
-- [Blog article, detailing our model launch](https://blog.rwkv.com/p/rwkv-v6-finch-14b-is-here)
+- [Featherless Inference](https://featherless.ai/models/RWKV/)
 
 ## Acknowledgement
 We are grateful for the help and support from the following key groups: