Update README.md

README.md

@@ -12,4 +12,104 @@ tags:
 - sentiment_analysis
 - custom_code
 - hf_integration
----
+---
+
+# MinGRU Sentiment Analysis
+
+![minGRU](minGRU.jpg)
+
+First Hugging Face integration of minGRU models from the paper "[**Were RNNs All We Needed?**](https://arxiv.org/abs/2410.01201)".
+
+This model uses BERT-Base-Uncased tokenizer.
+
+For modeling and configuration codes: [**minGRU-hf**](https://github.com/suayptalha/minGRU-hf/tree/main)
+
+# Example Usage:
+```py
+from transformers import AutoModelForSequenceClassification
+model = AutoModelForSequenceClassification.from_pretrained(
+    "suayptalha/minGRU-Sentiment-Analysis",
+    trust_remote_code = True
+).to("cuda")
+
+tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
+
+text = "The movie was absolutely wonderful, I loved it!"
+
+inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=128).to(device)
+
+with torch.no_grad():
+    outputs = model(**inputs)
+    logits = outputs.logits
+    prediction = torch.argmax(logits, dim=-1).item()
+
+sentiment = "positive" if prediction == 1 else "negative"
+print(f"Text: {text}")
+print(f"Predicted sentiment: {sentiment}")
+```
+
+> Text: The movie was absolutely wonderful, I loved it!
+> Predicted sentiment: positive
+
+# Training:
+
+Training code:
+
+```py
+from torch.optim import AdamW
+from torch.nn import CrossEntropyLoss
+import matplotlib.pyplot as plt
+from tqdm import tqdm
+
+optimizer = AdamW(model.parameters(), lr=5e-5)
+criterion = CrossEntropyLoss()
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+model.to(device)
+
+num_epochs = 5
+loss_values = []
+
+for epoch in range(num_epochs):
+    model.train()
+    epoch_loss = 0
+    progress_bar = tqdm(train_dataloader, desc=f"Epoch {epoch + 1}")
+
+    for batch in progress_bar:
+        input_ids = batch["input_ids"].to(device)
+        labels = batch["label"].to(device)
+
+        optimizer.zero_grad()
+        outputs = model(input_ids=input_ids, labels=labels)
+        loss = outputs.loss
+        loss.backward()
+        optimizer.step()
+
+        epoch_loss += loss.item()
+        progress_bar.set_postfix(loss=epoch_loss / len(progress_bar))
+    
+    avg_loss = epoch_loss / len(progress_bar)
+    loss_values.append(avg_loss)
+
+# Loss Graph
+plt.figure(figsize=(10, 6))
+plt.plot(range(1, num_epochs + 1), loss_values, marker='o', label='Training Loss')
+plt.xlabel("Epoch")
+plt.ylabel("Loss")
+plt.title("Training Loss Over Epochs")
+plt.legend()
+plt.grid(True)
+plt.show()
+```
+
+You can use this code snippet for fine-tuning!
+
+# Loss Graph:
+
+![Loss Graph](loss.png)
+
+# Credits:
+
+https://arxiv.org/abs/2410.01201
+
+I am thankful to Leo Feng, Frederick Tung, Mohamed Osama Ahmed, Yoshua Bengio and Hossein Hajimirsadeghi for their papers.