File size: 3,234 Bytes
489a9af 4f3b3bd bab0761 66cc42c 5b43bab 33c9ce8 489a9af 40fda82 60ee3af 40fda82 60ee3af 40fda82 60ee3af 40fda82 60ee3af 40fda82 60ee3af 40fda82 60ee3af 40fda82 9137016 40fda82 3b860ab bdeec8b 3b860ab 09e3be4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
---
metrics:
- accuracy
pipeline_tag: text-classification
tags:
- code
datasets:
- shukdevdatta123/twitter_sentiment_preprocessed
language:
- en
base_model: distilbert/distilbert-base-uncased
library_name: transformers
license: cc-by-nd-4.0
---
# DistilBERT-base-uncased LoRA Text Classification Model
## Model Description
This model is a fine-tuned version of `distilbert-base-uncased` on an unspecified dataset. It achieves the following results on the evaluation set:
- **Loss:** 0.4649
- **Accuracy:** 84.16%
## Intended Uses & Limitations
This is a text-classification based model.
## Training and Evaluation Data
Look below for more details about the performances.
## Steps to follow
- Installing the Libraries
- Loading the Dataset from HuggingFace
- Train_test Split the Dataset
- Model
- Preprocess Data
- Evaluation
- Apply untrained base model("distilbert-base-uncased") to text
- Train Model using LoRA
- Generate Prediction
- Save the Model and the Tokenizer
- Load the Model and the Tokenizer to test
- Push Model to HuggingFaceHub
### Training Hyperparameters
The following hyperparameters were used during training:
- **Learning Rate:** 0.001
- **Train Batch Size:** 4
- **Eval Batch Size:** 4
- **Seed:** 42
- **Optimizer:** Adam with betas=(0.9,0.999) and epsilon=1e-08
- **LR Scheduler Type:** Linear
- **Number of Epochs:** 10
### Training Results
| Epoch | Training Loss | Validation Loss | Validation Accuracy |
|-------|---------------|-----------------|---------------------|
| 1.0 | 0.5924 | 0.5523 | 78.45% |
| 2.0 | 0.5983 | 0.5236 | 80.29% |
| 3.0 | 0.5703 | 0.4498 | 79.56% |
| 4.0 | 0.5526 | 0.4976 | 80.66% |
| 5.0 | 0.5326 | 0.4317 | 80.85% |
| 6.0 | 0.5851 | 0.4562 | 82.87% |
| 7.0 | 0.5466 | 0.4713 | 81.95% |
| 8.0 | 0.5494 | 0.5072 | 82.50% |
| 9.0 | 0.5748 | 0.4802 | 82.87% |
| 10.0 | 0.5001 | 0.4649 | 84.16% |
## Framework Versions
- **PEFT:** 0.12.0
- **Transformers:** 4.42.4
- **PyTorch:** 2.4.0+cu121
- **Datasets:** 2.21.0
- **Tokenizers:** 0.19.1
# Dataset Viewer
You can view the dataset using the following link:
[View Twitter Sentiment Preprocessed Dataset](https://huggingface.co/datasets/shukdevdatta123/twitter_sentiment_preprocessed/)
Simply click the link to open the dataset viewer in your browser.
# Model Viewer
You can view the model using the following link:
[View Model in HuggingFace](https://huggingface.co/shukdevdatta123/distilbert-base-uncased-lora-text-classification/)
Simply click the link to open the model file in your browser.
Check out the "Fine-tune LLM.pptx" file for the theory behind this code.
# Github Repository
You can view the github using the following link:
[View GitHub Repository](https://github.com/shukdevtroy/Fine-Tune-LLM-using-LoRA-on-custom-dataset/)
Simply click the link to open the github repo in your browser.
Check out the "Fine-tune LLM.pptx" file in the GitHub repo for the theory behind this code.
|