metadata
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
datasets:
- cerebras/SlimPajama-627B
- bigcode/starcoderdata
- HuggingFaceH4/ultrachat_200k
- HuggingFaceH4/ultrafeedback_binarized
language:
- en
license: apache-2.0
tags:
- openvino
- nncf
- 4-bit
widget:
- example_title: Fibonacci (Python)
messages:
- role: system
content: You are a chatbot who can help code!
- role: user
content: >-
Write me a function to calculate the first 10 digits of the fibonacci
sequence in Python and print it out to the CLI.
This model is a quantized version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 and is converted to the OpenVINO format. This model was obtained via the nncf-quantization space with optimum-intel.
First make sure you have optimum-intel installed:
pip install optimum[openvino]
To load your model you can do as follows:
from optimum.intel import OVModelForCausalLM
model_id = "AlexKoff88/TinyLlama-1.1B-Chat-v1.0-openvino-4bit"
model = OVModelForCausalLM.from_pretrained(model_id)