Text Generation
Transformers
Safetensors
PyTorch
mistral
finetuned
quantized
4-bit precision
AWQ
instruct
conversational
text-generation-inference
finetune
chatml
awq
Suparious commited on
Commit
4b2d2b4
·
1 Parent(s): 9af97ed

Update README with model description.

Browse files
Files changed (1) hide show
  1. README.md +37 -0
README.md CHANGED
@@ -1,3 +1,40 @@
1
  ---
 
2
  license: cc-by-nc-4.0
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ base_model: NeverSleep/Noromaid-7B-0.4-DPO
3
  license: cc-by-nc-4.0
4
+ model_creator: IkariDev and Undi
5
+ model_name: Noromaid 7B v0.4 DPO
6
+ model_type: mistral
7
+ prompt_template: <|im_start|>system
8
+ You are Noromaid, a helpful AI companion.<|im_end|>
9
+ <|im_start|>user
10
+ {prompt}<|im_end|>
11
+ <|im_start|>assistant
12
+ quantized_by: Suparious
13
  ---
14
+
15
+ # Noromaid 7B v0.4 DPO - AWQ
16
+
17
+ - Model creator: [IkariDev and Undi](https://huggingface.co/NeverSleep)
18
+ - Original model: [Noromaid 13B v0.3](https://huggingface.co/NeverSleep/Noromaid-7B-0.4-DPO)
19
+
20
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/630dfb008df86f1e5becadc3/VKX2Z2yjZX5J8kXzgeCYO.png)
21
+
22
+ ## Model description
23
+
24
+ This repo contains AWQ model files for [IkariDev and Undi's Noromaid 7B v0.4 DPO](https://huggingface.co/NeverSleep/Noromaid-7B-0.4-DPO).
25
+
26
+ These files were quantised using hardware kindly provided by [SolidRusT Networks](https://solidrust.net/).
27
+
28
+ ### About AWQ
29
+
30
+ AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference with equivalent or better quality compared to the most commonly used GPTQ settings.
31
+
32
+ AWQ models are currently supported on Linux and Windows, with NVidia GPUs only. macOS users: please use GGUF models instead.
33
+
34
+ It is supported by:
35
+
36
+ - [Text Generation Webui](https://github.com/oobabooga/text-generation-webui) - using Loader: AutoAWQ
37
+ - [vLLM](https://github.com/vllm-project/vllm) - version 0.2.2 or later for support for all model types.
38
+ - [Hugging Face Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference)
39
+ - [Transformers](https://huggingface.co/docs/transformers) version 4.35.0 and later, from any code or client that supports Transformers
40
+ - [AutoAWQ](https://github.com/casper-hansen/AutoAWQ) - for use from Python code