Update README.md
Browse files
README.md
CHANGED
@@ -14,7 +14,7 @@ base_model:
|
|
14 |
---
|
15 |
# Model Card for Granite-3.0-8b-lab-community [Paper](https://arxiv.org/abs/2403.01081)
|
16 |
|
17 |
-
# NOTE
|
18 |
|
19 |
|
20 |
### Overview
|
@@ -26,24 +26,6 @@ Granite-3.0-8b-lab-community is the first release in our community-driven, openl
|
|
26 |
|
27 |

|
28 |
|
29 |
-
### Performance
|
30 |
-
|
31 |
-
| Model | Alignment | Base | Teacher | MTBench (Avg) * | MMLU(5-shot) |
|
32 |
-
| --- | --- | --- | --- | --- | --- |
|
33 |
-
| [Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) | RLHF | Llama-2-13b | Human Annotators | 6.65 |54.58 |
|
34 |
-
| [Orca-2-13b](https://huggingface.co/microsoft/Orca-2-13b) | Progressive Training | Llama-2-13b | GPT-4 | 6.15 | 60.37 * |
|
35 |
-
| [WizardLM-13B-V1.2](https://huggingface.co/WizardLM/WizardLM-13B-V1.2) | Evol-Instruct | Llama-2-13b | GPT-4 | 7.20 | 54.83 |
|
36 |
-
| [Labradorite-13b](https://huggingface.co/ibm/labradorite-13b) | Large-scale Alignment for chatBots (LAB) | Llama-2-13b | Mixtral-8x7B-Instruct | 7.23 | 58.89 |
|
37 |
-
| [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) | SFT | Mistral-7B-v0.1 | - | 6.84 | 60.37 |
|
38 |
-
| [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) | SFT/DPO | Mistral-7B-v0.1 | GPT-4 | 7.34 | 61.07 |
|
39 |
-
| [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) | SFT | Mistral-7B-v0.1 | - | 7.6** | 60.78 |
|
40 |
-
| [Merlinite-7b-lab](https://huggingface.co/instructlab/merlinite-7b-lab) | Large-scale Alignment for chatBots (LAB) | Mistral-7B-v0.1 | Mixtral-8x7B-Instruct | 7.66 |64.88 |
|
41 |
-
| Granite-7b-lab | Large-scale Alignment for chatBots (LAB) | Granite-7b-base| Mixtral-8x7B-Instruct | 6.69 | 51.91 |
|
42 |
-
|
43 |
-
[*] Numbers for models other than Merlinite-7b-lab, Granite-7b-lab and [Labradorite-13b](https://huggingface.co/ibm/labradorite-13b) are taken from [lmsys/chatbot-arena-leaderboard](https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard)
|
44 |
-
|
45 |
-
[**] Numbers taken from [MistralAI Release Blog](https://mistral.ai/news/la-plateforme/)
|
46 |
-
|
47 |
### Method
|
48 |
|
49 |
LAB: **L**arge-scale **A**lignment for chat**B**ots is a novel synthetic data-based alignment tuning method for LLMs from IBM Research. Granite-7b-lab is a Granite-7b-base derivative model trained with the LAB methodology, using Mixtral-8x7b-Instruct as a teacher model.
|
@@ -81,16 +63,16 @@ Importantly, we use a set of hyper-parameters for training that are very differe
|
|
81 |

|
82 |
|
83 |
## Model description
|
84 |
-
- **Model Name**: Granite-
|
85 |
- **Language(s):** Primarily English
|
86 |
- **License:** Apache 2.0
|
87 |
-
- **Base model:** [ibm/granite-
|
88 |
- **Teacher Model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
|
89 |
|
90 |
## Prompt Template
|
91 |
|
92 |
```python
|
93 |
-
sys_prompt = "
|
94 |
|
95 |
prompt = f'<|system|>\n{sys_prompt}\n<|user|>\n{inputs}\n<|assistant|>\n'
|
96 |
stop_token = '<|endoftext|>'
|
@@ -102,4 +84,4 @@ We advise utilizing the system prompt employed during the model's training for o
|
|
102 |
|
103 |
**Bias, Risks, and Limitations**
|
104 |
|
105 |
-
Granite-
|
|
|
14 |
---
|
15 |
# Model Card for Granite-3.0-8b-lab-community [Paper](https://arxiv.org/abs/2403.01081)
|
16 |
|
17 |
+
# NOTE: MODEL CARD UPDATES IN-PROGRESS
|
18 |
|
19 |
|
20 |
### Overview
|
|
|
26 |
|
27 |

|
28 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
29 |
### Method
|
30 |
|
31 |
LAB: **L**arge-scale **A**lignment for chat**B**ots is a novel synthetic data-based alignment tuning method for LLMs from IBM Research. Granite-7b-lab is a Granite-7b-base derivative model trained with the LAB methodology, using Mixtral-8x7b-Instruct as a teacher model.
|
|
|
63 |

|
64 |
|
65 |
## Model description
|
66 |
+
- **Model Name**: Granite-3.0-8b-lab-community
|
67 |
- **Language(s):** Primarily English
|
68 |
- **License:** Apache 2.0
|
69 |
+
- **Base model:** [ibm-granite/granite-3.0-8b-base](https://huggingface.co/ibm-granite/granite-3.0-8b-base)
|
70 |
- **Teacher Model:** [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
|
71 |
|
72 |
## Prompt Template
|
73 |
|
74 |
```python
|
75 |
+
sys_prompt = "I am a Red Hat® Instruct Model, an AI language model developed by Red Hat and IBM Research based on the granite-3.0-8b-base model. My primary role is to serve as a chat assistant."
|
76 |
|
77 |
prompt = f'<|system|>\n{sys_prompt}\n<|user|>\n{inputs}\n<|assistant|>\n'
|
78 |
stop_token = '<|endoftext|>'
|
|
|
84 |
|
85 |
**Bias, Risks, and Limitations**
|
86 |
|
87 |
+
Granite-3.0-8b-lab-community is a base model and has not undergone any safety alignment, there it may produce problematic outputs. In the absence of adequate safeguards and RLHF, there exists a risk of malicious utilization of these models for generating disinformation or harmful content. Caution is urged against complete reliance on a specific language model for crucial decisions or impactful information, as preventing these models from fabricating content is not straightforward. Additionally, it remains uncertain whether smaller models might exhibit increased susceptibility to hallucination in ungrounded generation scenarios due to their reduced sizes and memorization capacities. This aspect is currently an active area of research, and we anticipate more rigorous exploration, comprehension, and mitigations in this domain.
|