--- title: README emoji: πŸš€ colorFrom: blue colorTo: blue sdk: static pinned: false ---

Advancing Open-source Language Models with Mixed-Quality Data

OpenChat Logo Online Demo | GitHub Logo GitHub | ArXiv Logo Paper | Discord Logo Discord

OPENCHAT3.5 1210
πŸ† The Overall Best Performing Open Source 7B Model πŸ†
πŸ€– Outperforms ChatGPT (March) and Grok-1 πŸ€–
πŸš€15-point improvement in Coding over OpenChat-3.5πŸš€

New Features
πŸ’‘ 2 Modes: Coding + Generalist, Mathematical Reasoning πŸ’‘
πŸ§‘β€βš–οΈ Experimental support for Evaluator and Feedback capabilities πŸ§‘β€βš–οΈ

OpenChat LogoAbout OpenChat

- OpenChat is an innovative library of **open-source language models**, fine-tuned with [**C-RLFT**](https://arxiv.org/pdf/2309.11235.pdf) - a strategy inspired by offline reinforcement learning. - Our models learn from mixed-quality data without preference labels, delivering exceptional performance on par with `ChatGPT`, even with a `7B` model which can be run on a **consumer GPU (e.g. RTX 3090)**. - Despite our simple approach, we are committed to developing a high-performance, commercially viable, open-source large language model, and we continue to make significant strides toward this vision. # πŸ“° News - [2023/12/10] We released the [OpenChat-3.5-1210](https://huggingface.co/openchat/openchat-3.5-1210) model, 15-point improvements in coding. - [2023/11/01] We released the [OpenChat-3.5-7B](https://huggingface.co/openchat/openchat_3.5) model, surpassing ChatGPT on various benchmarks πŸ”₯. - [2023/09/21] We released our paper [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://arxiv.org/pdf/2309.11235.pdf). # πŸ“Š Benchmarks | Model | # Params | Average | MT-Bench | HumanEval | BBH MC | AGIEval | TruthfulQA | MMLU | GSM8K | BBH CoT | |--------------------|----------|----------|--------------|-----------------|----------|----------|---------------|--------------|--------------|-------------| | OpenChat-3.5-1210 | **7B** | **63.8** | 7.76 | **68.9** | **49.5** | **48.0** | **61.8** | 65.3 | **77.3** | 61.8 | | OpenChat-3.5 | **7B** | 61.6 | 7.81 | 55.5 | 47.6 | 47.4 | 59.1 | 64.3 | **77.3** | 63.5 | | ChatGPT (March)* | ? | 61.5 | **7.94** | 48.1 | 47.6 | 47.1 | 57.7 | **67.3** | 74.9 | **70.1** | | | | | | | | | | | | | | OpenHermes 2.5 | 7B | 59.3 | 7.54 | 48.2 | 49.4 | 46.5 | 57.5 | 63.8 | 73.5 | 59.9 | | OpenOrca Mistral | 7B | 52.7 | 6.86 | 38.4 | 49.4 | 42.9 | 45.9 | 59.3 | 59.1 | 58.1 | | Zephyr-Ξ²^ | 7B | 34.6 | 7.34 | 22.0 | 40.6 | 39.0 | 40.8 | 39.8 | 5.1 | 16.0 | | Mistral | 7B | - | 6.84 | 30.5 | 39.0 | 38.0 | - | 60.1 | 52.2 | - | ## 𝕏 Comparison with [X.AI Grok](https://x.ai/) | | License | # Param | Average | MMLU | HumanEval | MATH | GSM8k | |-------------------|-------------|---------|----------|------|-----------|----------|----------| | OpenChat 3.5 1210 | Apache-2.0 | **7B** | **60.1** | 65.3 | **68.9** | **28.9** | **77.3** | | OpenChat 3.5 | Apache-2.0 | **7B** | 56.4 | 64.3 | 55.5 | 28.6 | **77.3** | | Grok-0 | Proprietary | 33B | 44.5 | 65.7 | 39.7 | 15.7 | 56.8 | | Grok-1 | Proprietary | ???B | 55.8 | 73 | 63.2 | 23.9 | 62.9 | # πŸ’ŒContact We are a student team working on OpenChat, a project that requires additional computing power or LLMs API keys for further development. If you are interested in our project and would like to offer support, please feel free to reach out to us: * Wang Guan [imonenext@gmail.com] * Cheng Sijie [csj23@mails.tsinghua.edu.cn] * Alpay Ariyak [aariyak@wpi.edu] We look forward to hearing you and collaborating on this exciting project!