README.md

---
license: mit
pipeline_tag: summarization
---

# ANNOUNCEMENT:

!!! The previous pipeline has been destroyed by a system update of hugging face. !!!
In the updated pipeline, the generated text may show like: 
```
Today was a beautiful day, I have done all my homework. I wonder is this I need? Thank you so much for your help.TL;DRThank You
```
please filter the result by TL;DR.

## Model Description
This is the project for the course Multimedia System in Leiden University 2023 fall. The model is part of self-designed software for the final project that summarizes short texts.
The original summarization model is from https://towardsdatascience.com/text-summarization-with-gpt2-and-layer-ai-599625085d8e .

## Training Datasets
- Amazon review dataset https://www.kaggle.com/datasets/kritanjalijain/amazon-reviews?select=amazon_review_polarity_csv.tgz
- Twitter crawler dataset https://www.kaggle.com/datasets/tripathiharsh/training
- Emotion analysis dataset https://huggingface.co/datasets/dair-ai/emotion
- Kindle review dataset https://www.kaggle.com/datasets/meetnagadia/amazon-kindle-book-review-for-sentiment-analysis/data

## TODO
- Improve training data quality
- Model analysis

## Notification
Due to the technique support of hugging face, this project can only functioning with Interface code below. When you first call the API, it may take more than 1 minutes to download the model and functioning. Please wait carefully, thank you!

## Fine-tune Code
You can check "text_summary.py" in the uploaded documents, and download the model to test and modify this task.

## Interface(legacy)
please see the announcement above.
```python
import requests

API_URL = "https://api-inference.huggingface.co/models/Lin0He/text-summary-gpt2-short"
headers = {"Authorization": "Bearer <Your API tokens>"}

def query(payload):
    response = requests.post(API_URL, headers=headers, json=payload)
    return response.json()
    
output = query({
    "inputs": "Today was a beautiful day. I have down all my homework, and baked some cookies with my friends. What a lovely day."
})
print(output)
```