uploaded readme

README.md

@@ -0,0 +1,200 @@
+Quantization made by Richard Erkhov.
+
+[Github](https://github.com/RichardErkhov)
+
+[Discord](https://discord.gg/pvy7H8DZMG)
+
+[Request more models](https://github.com/RichardErkhov/quant_request)
+
+
+PowerMoE-3b - GGUF
+- Model creator: https://huggingface.co/ibm/
+- Original model: https://huggingface.co/ibm/PowerMoE-3b/
+
+
+| Name | Quant method | Size |
+| ---- | ---- | ---- |
+| [PowerMoE-3b.Q2_K.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q2_K.gguf) | Q2_K | 1.18GB |
+| [PowerMoE-3b.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.IQ3_XS.gguf) | IQ3_XS | 1.32GB |
+| [PowerMoE-3b.IQ3_S.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.IQ3_S.gguf) | IQ3_S | 1.39GB |
+| [PowerMoE-3b.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q3_K_S.gguf) | Q3_K_S | 1.39GB |
+| [PowerMoE-3b.IQ3_M.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.IQ3_M.gguf) | IQ3_M | 1.41GB |
+| [PowerMoE-3b.Q3_K.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q3_K.gguf) | Q3_K | 1.53GB |
+| [PowerMoE-3b.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q3_K_M.gguf) | Q3_K_M | 1.53GB |
+| [PowerMoE-3b.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q3_K_L.gguf) | Q3_K_L | 1.65GB |
+| [PowerMoE-3b.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.IQ4_XS.gguf) | IQ4_XS | 1.72GB |
+| [PowerMoE-3b.Q4_0.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q4_0.gguf) | Q4_0 | 1.79GB |
+| [PowerMoE-3b.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.IQ4_NL.gguf) | IQ4_NL | 1.81GB |
+| [PowerMoE-3b.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q4_K_S.gguf) | Q4_K_S | 1.81GB |
+| [PowerMoE-3b.Q4_K.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q4_K.gguf) | Q4_K | 1.92GB |
+| [PowerMoE-3b.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q4_K_M.gguf) | Q4_K_M | 1.92GB |
+| [PowerMoE-3b.Q4_1.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q4_1.gguf) | Q4_1 | 1.99GB |
+| [PowerMoE-3b.Q5_0.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q5_0.gguf) | Q5_0 | 2.18GB |
+| [PowerMoE-3b.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q5_K_S.gguf) | Q5_K_S | 2.18GB |
+| [PowerMoE-3b.Q5_K.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q5_K.gguf) | Q5_K | 2.24GB |
+| [PowerMoE-3b.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q5_K_M.gguf) | Q5_K_M | 2.24GB |
+| [PowerMoE-3b.Q5_1.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q5_1.gguf) | Q5_1 | 2.37GB |
+| [PowerMoE-3b.Q6_K.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q6_K.gguf) | Q6_K | 2.59GB |
+| [PowerMoE-3b.Q8_0.gguf](https://huggingface.co/RichardErkhov/ibm_-_PowerMoE-3b-gguf/blob/main/PowerMoE-3b.Q8_0.gguf) | Q8_0 | 3.35GB |
+
+
+
+
+Original model description:
+---
+pipeline_tag: text-generation
+inference: false
+license: apache-2.0
+library_name: transformers
+model-index:
+- name: ibm/PowerMoE-3b
+  results:
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: ARC
+    metrics:
+    - name: accuracy-norm
+      type: accuracy-norm
+      value: 58.1
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: BoolQ
+    metrics:
+    - name: accuracy
+      type: accuracy
+      value: 65.0
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: Hellaswag
+    metrics:
+    - name: accuracy-norm
+      type: accuracy-norm
+      value: 71.5
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: OpenBookQA
+    metrics:
+    - name: accuracy-norm
+      type: accuracy-norm
+      value: 41.0
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: PIQA
+    metrics:
+    - name: accuracy-norm
+      type: accuracy-norm
+      value: 79.1
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: Winogrande
+    metrics:
+    - name: accuracy-norm
+      type: accuracy-norm
+      value: 65.0
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: MMLU (5 shot)
+    metrics:
+    - name: accuracy
+      type: accuracy
+      value: 42.8
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: GSM8k (5 shot)
+    metrics:
+    - name: accuracy
+      type: accuracy
+      value: 25.9
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: lm-eval-harness
+      name: math (4 shot)
+    metrics:
+    - name: accuracy
+      type: accuracy
+      value: 14.8
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: bigcode-eval
+      name: humaneval
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 20.1
+      verified: false
+  - task:
+      type: text-generation
+    dataset:
+      type: bigcode-eval
+      name: MBPP
+    metrics:
+    - name: pass@1
+      type: pass@1
+      value: 32.4
+      verified: false
+---
+
+## Model Summary
+PowerMoE-3B is a 3B sparse Mixture-of-Experts (sMoE) language model trained with the Power learning rate scheduler. It sparsely activates 800M parameters for each token. It is trained on a mix of open-source and proprietary datasets. PowerMoE-3B has shown promising results compared to other dense models with 2x activate parameters across various benchmarks, including natural language multi-choices, code generation, and math reasoning.
+Paper: https://arxiv.org/abs/2408.13359
+
+## Usage
+Note: Requires installing HF transformers from source.
+
+### Generation
+This is a simple example of how to use **PowerMoE-3b** model.
+
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+device = "cuda" # or "cpu"
+model_path = "ibm/PowerMoE-3b"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+# drop device_map if running on CPU
+model = AutoModelForCausalLM.from_pretrained(model_path, device_map=device)
+model.eval()
+# change input text as desired
+prompt = "Write a code to find the maximum value in a list of numbers."
+# tokenize the text
+input_tokens = tokenizer(prompt, return_tensors="pt")
+# transfer tokenized inputs to the device
+for i in input_tokens:
+    input_tokens[i] = input_tokens[i].to(device)
+# generate output tokens
+output = model.generate(**input_tokens, max_new_tokens=100)
+# decode output tokens into text
+output = tokenizer.batch_decode(output)
+# loop over the batch to print, in this example the batch size is 1
+for i in output:
+    print(i)
+```
+
+
+Additional thanks to @nicoboss for giving me access to his private supercomputer, enabling me to provide many more quants, at much higher speed, than I would otherwise be able to.