Create README.md

README.md

@@ -0,0 +1,30 @@
+---
+tags:
+- deepsparse
+---
+## Usage
+
+```python
+from deepsparse import TextGeneration
+
+prompt = "How to get in a good university?"
+formatted_prompt =  f"<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n"
+
+model = TextGeneration(model="hf:nm-testing/TinyLlama-1.1B-Chat-v0.4-pruned50-quant")
+print(model(formatted_prompt, max_new_tokens=200).generations[0].text)
+
+"""
+The university is a place where you learn and gain skills. It is a place where you learn by reading books, doing experiments, and doing research. It is a place where you learn by doing projects. It is a place where you learn by doing research. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a place where you learn by doing research. The university is a place where you learn by reading books. The university is a place where you learn by doing projects. The university is a
+"""
+```
+## One-shot and Export
+
+```bash
+git clone https://github.com/neuralmagic/sparseml
+pip install -e "sparseml[transformers]" "torch<2"
+git checkout update/onnx_export/duplicate
+python sparseml/src/sparseml/transformers/sparsification/obcq/obcq.py TinyLlama/TinyLlama-1.1B-Chat-v0.4 open_platypus --recipe recipe.yaml --save True
+python sparseml/src/sparseml/transformers/sparsification/obcq/export.py --task text-generation --model_path obcq_deployment 
+cp deployment/model.onnx deployment/model-orig.onnx
+python onnx_kv_inject.py --input-file deployment/model-orig.onnx --output-file deployment/model.onnx
+```