Description

This model was obtained by fine-tuning the Llama-2 7B large language model with the LoRA technique. The aim is to develop a sentiment analysis system in Turkish language by training the model according to the sentences in the given data set. The evaluation metrics of the model were calculated and the following results were obtained.

Dataset

The training data set consists of 152715 rows and the eval data set consists of 16968 rows. It includes social media posts and product reviews.

Uses

from transformers import AutoConfig from transformers import AutoModelForSequenceClassification

config = AutoConfig.from_pretrained("Minekorkmz/model_yurt_1200") num_labels = config.num_labels

base_model = AutoModelForSequenceClassification.from_pretrained( "meta-llama/Llama-2-7b-chat-hf", num_labels=num_labels )

model = PeftModel.from_pretrained(base_model, "Minekorkmz/model_yurt_1200") tokenizer = AutoTokenizer.from_pretrained("Minekorkmz/model_yurt_1200")

from transformers import pipeline

sentiment_task = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer, return_all_scores=True)

print(sentiment_task("çok kötü bir ürün oldu sevemedim"))

Training procedure

The following bitsandbytes quantization config was used during training:

  • quant_method: bitsandbytes
  • load_in_8bit: False
  • load_in_4bit: True
  • llm_int8_threshold: 6.0
  • llm_int8_skip_modules: None
  • llm_int8_enable_fp32_cpu_offload: False
  • llm_int8_has_fp16_weight: False
  • bnb_4bit_quant_type: nf4
  • bnb_4bit_use_double_quant: True
  • bnb_4bit_compute_dtype: float16

Framework versions

  • PEFT 0.4.0
  • accelerate 0.26.0
  • bitsandbytes 0.41.1
  • transformers 4.35.0
  • trl 0.4.7
Downloads last month
1
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.