Suparious commited on
Commit
3c098a8
·
verified ·
1 Parent(s): 585e728

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -2
README.md CHANGED
@@ -1,4 +1,13 @@
1
  ---
 
 
 
 
 
 
 
 
 
2
  library_name: transformers
3
  tags:
4
  - 4-bit
@@ -10,6 +19,17 @@ pipeline_tag: text-generation
10
  inference: false
11
  quantized_by: Suparious
12
  ---
13
- #
14
 
15
- **UPLOAD IN PROGRESS**
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: other
3
+ datasets:
4
+ - mlabonne/orpo-dpo-mix-40k
5
+ - Open-Orca/SlimOrca-Dedup
6
+ - jondurbin/airoboros-3.2
7
+ - microsoft/orca-math-word-problems-200k
8
+ - m-a-p/Code-Feedback
9
+ - MaziyarPanahi/WizardLM_evol_instruct_V2_196k
10
+ base_model: meta-llama/Meta-Llama-3-8B
11
  library_name: transformers
12
  tags:
13
  - 4-bit
 
19
  inference: false
20
  quantized_by: Suparious
21
  ---
22
+ # Locutusque/llama-3-neural-chat-v1-8b AWQ
23
 
24
+ - Model creator: [Locutusque](https://huggingface.co/Locutusque)
25
+ - Original model: [llama-3-neural-chat-v1-8b](https://huggingface.co/Locutusque/llama-3-neural-chat-v1-8b)
26
+
27
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6437292ecd93f4c9a34b0d47/6XQuhjWNr6C4RbU9f1k99.png)
28
+
29
+ ## Model Summary
30
+
31
+ OpenHermes 2.5 Mistral 7B is a state of the art Mistral Fine-tune, a continuation of OpenHermes 2 model, which trained on additional code datasets.
32
+
33
+ Potentially the most interesting finding from training on a good ratio (est. of around 7-14% of the total dataset) of code instruction was that it has boosted several non-code benchmarks, including TruthfulQA, AGIEval, and GPT4All suite. It did however reduce BigBench benchmark score, but the net gain overall is significant.
34
+
35
+ Here, we are finetuning openheremes using DPO with various data meant to improve its abilities.