Hugging Face's logo

tensorblock
/
TeenyTinyLlama-460m-Chat-GGUF

Text Generation
Transformers
GGUF
Portuguese
alignment
instruction tuned
text generation
conversation
assistant
TensorBlock
GGUF
Inference Endpoints
Model card Files Community
TeenyTinyLlama-460m-Chat-GGUF / README.md
morriszms's picture
morriszms
Upload folder using huggingface_hub
f1f91df verified
preview code
|
raw
history blame contribute delete
6.18 kB
metadata
license: apache-2.0
datasets:
  - nicholasKluge/instruct-aira-dataset-v2
language:
  - pt
metrics:
  - accuracy
library_name: transformers
pipeline_tag: text-generation
tags:
  - alignment
  - instruction tuned
  - text generation
  - conversation
  - assistant
  - TensorBlock
  - GGUF
widget:
  - text: >-
      <s><instruction>Cite algumas bandas de rock famosas da década de
      1960.</instruction>
    example_title: Exemplo
  - text: <s><instruction>Quantos planetas existem no sistema solar?</instruction>
    example_title: Exemplo
  - text: <s><instruction>Qual é o futuro do ser humano?</instruction>
    example_title: Exemplo
  - text: <s><instruction>Qual o sentido da vida?</instruction>
    example_title: Exemplo
  - text: <s><instruction>Como imprimir hello world em python?</instruction>
    example_title: Exemplo
  - text: >-
      <s><instruction>Invente uma história sobre um encanador com poderes
      mágicos.</instruction>
    example_title: Exemplo
inference:
  parameters:
    repetition_penalty: 1.2
    temperature: 0.2
    top_k: 30
    top_p: 0.3
    max_new_tokens: 200
    length_penalty: 0.3
    early_stopping: true
co2_eq_emissions:
  emissions: 2530
  source: CodeCarbon
  training_type: fine-tuning
  geographical_location: United States of America
  hardware_used: NVIDIA A100-SXM4-40GB
base_model: nicholasKluge/TeenyTinyLlama-460m-Chat
TensorBlock

Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server

nicholasKluge/TeenyTinyLlama-460m-Chat - GGUF

This repo contains GGUF format model files for nicholasKluge/TeenyTinyLlama-460m-Chat.

The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4242.

Prompt template

Model file specification

Filename Quant type File Size Description
TeenyTinyLlama-460m-Chat-Q2_K.gguf Q2_K 0.186 GB smallest, significant quality loss - not recommended for most purposes
TeenyTinyLlama-460m-Chat-Q3_K_S.gguf Q3_K_S 0.215 GB very small, high quality loss
TeenyTinyLlama-460m-Chat-Q3_K_M.gguf Q3_K_M 0.236 GB very small, high quality loss
TeenyTinyLlama-460m-Chat-Q3_K_L.gguf Q3_K_L 0.254 GB small, substantial quality loss
TeenyTinyLlama-460m-Chat-Q4_0.gguf Q4_0 0.273 GB legacy; small, very high quality loss - prefer using Q3_K_M
TeenyTinyLlama-460m-Chat-Q4_K_S.gguf Q4_K_S 0.275 GB small, greater quality loss
TeenyTinyLlama-460m-Chat-Q4_K_M.gguf Q4_K_M 0.289 GB medium, balanced quality - recommended
TeenyTinyLlama-460m-Chat-Q5_0.gguf Q5_0 0.327 GB legacy; medium, balanced quality - prefer using Q4_K_M
TeenyTinyLlama-460m-Chat-Q5_K_S.gguf Q5_K_S 0.327 GB large, low quality loss - recommended
TeenyTinyLlama-460m-Chat-Q5_K_M.gguf Q5_K_M 0.336 GB large, very low quality loss - recommended
TeenyTinyLlama-460m-Chat-Q6_K.gguf Q6_K 0.385 GB very large, extremely low quality loss
TeenyTinyLlama-460m-Chat-Q8_0.gguf Q8_0 0.498 GB very large, extremely low quality loss - not recommended

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, downoad the individual model file the a local directory

huggingface-cli download tensorblock/TeenyTinyLlama-460m-Chat-GGUF --include "TeenyTinyLlama-460m-Chat-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/TeenyTinyLlama-460m-Chat-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'