Create README.md

README.md

@@ -0,0 +1,32 @@
+---
+language:
+- en
+base_model:
+- google-t5/t5-large
+---
+Model: t5-DistillingSbS-ABSA
+
+Task: Aspect-Based Sentiment Analysis (ABSA) - specifically, Aspect Pair Sentiment Extraction
+
+Technique: Distilling Step-by-Step (DistillingSbS) 
+
+Model Description
+t5-DistillingSbS-ABSA is a fine-tuned t5-large model designed to perform Aspect-Based Sentiment Analysis (ABSA), particularly for the task of Aspect Pair Sentiment Extraction. 
+I used a training approach called Distilling Step-by-Step originally proposed in [This Paper](https://arxiv.org/abs/2305.02301) by Hsieh et al. at Google Research
+
+Data Annotation: Reviews were initially unannotated and were labeled using the OpenAI API for aspect-sentiment pairs and rationales.
+
+Dataset
+The dataset consists of customer reviews of mobile apps that were originally unannotated. The data was annotated via the OpenAI API, with each review labeled for specific aspects (e.g., UI, functionality, performance) and the corresponding sentiment (positive, negative, neutral). Additionally, sentence-long rationales were extracted to justify the aspect-sentiment pair annotations, aiding in Distilling Step-by-Step training.
+
+Training was performed using Hugging Face's Trainer API in Google Colaboratory using 1 A100 GPU with 40 GB of VRAM. 
+Training took around 6 hours with a cost of about 80 compute units. 
+With a custom loss function, tokenization function and training loop. All code can be found at my [My GitHub Repository](https://github.com/trichter93/ABSA-LLMs-DistillingSbS/)
+
+Hyperparameters
+Some of the key hyperparameters used for fine-tuning:
+
+Batch Size: 3
+Learning Rate: 1e-4
+Epochs: 5
+Max Sequence Length: 512