Update README.md
Browse files
README.md
CHANGED
|
@@ -7,135 +7,91 @@ tags:
|
|
| 7 |
- ViT
|
| 8 |
- detection
|
| 9 |
- Image
|
| 10 |
-
- transformers-4.49.0.dev0
|
| 11 |
base_model:
|
| 12 |
- google/vit-base-patch16-224-in21k
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
The **Deep-Fake-Detector-Model** is a state-of-the-art deep learning model designed to detect deepfake images. It leverages the **Vision Transformer (ViT)** architecture, specifically the `google/vit-base-patch16-224-in21k` model, fine-tuned on a dataset of real and deepfake images. The model is trained to classify images as either "Real" or "Fake" with high accuracy, making it a powerful tool for detecting manipulated media.
|
| 22 |
|
| 23 |
-
|
| 24 |
|
| 25 |
-
|
| 26 |
-
|------------|------|
|
| 27 |
-
| Deep Fake Detector Model | [GitHub Repository](https://github.com/PRITHIVSAKTHIUR/Deep-Fake-Detector-Model) |
|
| 28 |
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
- **Training Dataset**: A curated dataset of real and deepfake images.
|
| 34 |
-
- **Fine-Tuning**: The model is fine-tuned using Hugging Face's `Trainer` API with advanced data augmentation techniques.
|
| 35 |
-
- **Performance**: Achieves high accuracy and F1 score on validation and test datasets.
|
| 36 |
|
| 37 |
-
|
| 38 |
-
The model is based on the **Vision Transformer (ViT)**, which treats images as sequences of patches and applies a transformer encoder to learn spatial relationships. Key components include:
|
| 39 |
-
- **Patch Embedding**: Divides the input image into fixed-size patches (16x16 pixels).
|
| 40 |
-
- **Transformer Encoder**: Processes patch embeddings using multi-head self-attention mechanisms.
|
| 41 |
-
- **Classification Head**: A fully connected layer for binary classification.
|
| 42 |
|
| 43 |
-
|
| 44 |
-
- **Optimizer**: AdamW with a learning rate of `1e-6`.
|
| 45 |
-
- **Batch Size**: 32 for training, 8 for evaluation.
|
| 46 |
-
- **Epochs**: 2.
|
| 47 |
-
- **Data Augmentation**:
|
| 48 |
-
- Random rotation (±90 degrees).
|
| 49 |
-
- Random sharpness adjustment.
|
| 50 |
-
- Random resizing and cropping.
|
| 51 |
-
- **Loss Function**: Cross-Entropy Loss.
|
| 52 |
-
- **Evaluation Metrics**: Accuracy, F1 Score, and Confusion Matrix.
|
| 53 |
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
|
| 58 |
-
|
| 59 |
-
pipe = pipeline('image-classification', model="prithivMLmods/Deep-Fake-Detector-Model", device=0)
|
| 60 |
|
| 61 |
-
|
| 62 |
-
result = pipe("path_to_image.jpg")
|
| 63 |
-
print(result)
|
| 64 |
-
```
|
| 65 |
|
| 66 |
-
# **Inference with PyTorch**
|
| 67 |
```python
|
| 68 |
-
|
|
|
|
| 69 |
from PIL import Image
|
| 70 |
import torch
|
| 71 |
|
| 72 |
-
# Load
|
| 73 |
-
|
| 74 |
-
|
|
|
|
| 75 |
|
| 76 |
-
#
|
| 77 |
-
|
| 78 |
-
inputs = processor(images=image, return_tensors="pt")
|
| 79 |
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
logits = outputs.logits
|
| 84 |
-
predicted_class = torch.argmax(logits, dim=1).item()
|
| 85 |
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
# **Performance Metrics**
|
| 91 |
-
```
|
| 92 |
-
Classification report:
|
| 93 |
-
|
| 94 |
-
precision recall f1-score support
|
| 95 |
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
accuracy 0.6999 76134
|
| 100 |
-
macro avg 0.7935 0.7000 0.6739 76134
|
| 101 |
-
weighted avg 0.7936 0.6999 0.6739 76134
|
| 102 |
-
```
|
| 103 |
|
| 104 |
-
|
| 105 |
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
|
|
|
|
|
|
|
|
|
| 111 |
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
- **Fake Images**: Deepfake images generated using advanced AI techniques.
|
| 116 |
-
|
| 117 |
-
# **Limitations**
|
| 118 |
-
The model is trained on a specific dataset and may not generalize well to other deepfake datasets or domains.
|
| 119 |
-
- Performance may degrade on low-resolution or heavily compressed images.
|
| 120 |
-
- The model is designed for image classification and does not detect deepfake videos directly.
|
| 121 |
-
|
| 122 |
-
# **Ethical Considerations**
|
| 123 |
|
| 124 |
-
|
| 125 |
-
**Bias**: The model may inherit biases from the training dataset. Care should be taken to ensure fairness and inclusivity.
|
| 126 |
-
**Transparency**: Users should be informed when deepfake detection tools are used to analyze their content.
|
| 127 |
|
| 128 |
-
|
| 129 |
-
- Extend the model to detect deepfake videos.
|
| 130 |
-
- Improve generalization by training on larger and more diverse datasets.
|
| 131 |
-
- Incorporate explainability techniques to provide insights into model predictions.
|
| 132 |
|
| 133 |
-
|
| 134 |
|
| 135 |
-
|
| 136 |
-
|
| 137 |
-
|
| 138 |
-
|
| 139 |
-
|
| 140 |
-
last_updated = {31 Jan 2025}
|
| 141 |
-
}
|
|
|
|
| 7 |
- ViT
|
| 8 |
- detection
|
| 9 |
- Image
|
|
|
|
| 10 |
base_model:
|
| 11 |
- google/vit-base-patch16-224-in21k
|
| 12 |
+
datasets:
|
| 13 |
+
- prithivMLmods/OpenDeepfake-Preview
|
| 14 |
+
language:
|
| 15 |
+
- en
|
| 16 |
---
|
| 17 |
|
| 18 |
+
# deepfake-detector-model
|
| 19 |
|
| 20 |
+
> deepfake-detector-model is a vision-language model fine-tuned from `google/vit-base-patch16-224-in21k` for binary image classification. It is trained to detect whether an image is fake or real using the *OpenDeepfake-Preview* dataset. The model uses the `ViTForImageClassification` architecture.
|
| 21 |
|
| 22 |
+
---
|
|
|
|
|
|
|
| 23 |
|
| 24 |
+
## Label Space: 2 Classes
|
| 25 |
|
| 26 |
+
The model classifies an image as either:
|
|
|
|
|
|
|
| 27 |
|
| 28 |
+
```
|
| 29 |
+
Class 0: fake
|
| 30 |
+
Class 1: real
|
| 31 |
+
```
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
+
## Install Dependencies
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
+
```bash
|
| 38 |
+
pip install -q transformers torch pillow gradio hf_xet
|
| 39 |
+
```
|
| 40 |
|
| 41 |
+
---
|
|
|
|
| 42 |
|
| 43 |
+
## Inference Code
|
|
|
|
|
|
|
|
|
|
| 44 |
|
|
|
|
| 45 |
```python
|
| 46 |
+
import gradio as gr
|
| 47 |
+
from transformers import ViTImageProcessor, ViTForImageClassification
|
| 48 |
from PIL import Image
|
| 49 |
import torch
|
| 50 |
|
| 51 |
+
# Load model and processor
|
| 52 |
+
model_name = "your-username/deepfake-detector-model"
|
| 53 |
+
model = ViTForImageClassification.from_pretrained(model_name)
|
| 54 |
+
processor = ViTImageProcessor.from_pretrained(model_name)
|
| 55 |
|
| 56 |
+
# Updated label mapping
|
| 57 |
+
labels_list = ['fake', 'real']
|
|
|
|
| 58 |
|
| 59 |
+
def classify_image(image):
|
| 60 |
+
image = Image.fromarray(image).convert("RGB")
|
| 61 |
+
inputs = processor(images=image, return_tensors="pt")
|
|
|
|
|
|
|
| 62 |
|
| 63 |
+
with torch.no_grad():
|
| 64 |
+
outputs = model(**inputs)
|
| 65 |
+
logits = outputs.logits
|
| 66 |
+
probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
+
prediction = {
|
| 69 |
+
labels_list[i]: round(probs[i], 3) for i in range(len(probs))
|
| 70 |
+
}
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
|
| 72 |
+
return prediction
|
| 73 |
|
| 74 |
+
# Gradio Interface
|
| 75 |
+
iface = gr.Interface(
|
| 76 |
+
fn=classify_image,
|
| 77 |
+
inputs=gr.Image(type="numpy"),
|
| 78 |
+
outputs=gr.Label(num_top_classes=2, label="Deepfake Detection"),
|
| 79 |
+
title="deepfake-detector-model",
|
| 80 |
+
description="Upload an image to detect whether it is AI-generated (fake) or a real photograph (real), using the OpenDeepfake-Preview dataset."
|
| 81 |
+
)
|
| 82 |
|
| 83 |
+
if __name__ == "__main__":
|
| 84 |
+
iface.launch()
|
| 85 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
|
| 87 |
+
---
|
|
|
|
|
|
|
| 88 |
|
| 89 |
+
## Intended Use
|
|
|
|
|
|
|
|
|
|
| 90 |
|
| 91 |
+
`deepfake-detector-model` is designed for:
|
| 92 |
|
| 93 |
+
* **Deepfake Detection** – Identify AI-generated or manipulated images.
|
| 94 |
+
* **Content Moderation** – Flag synthetic or fake visual content.
|
| 95 |
+
* **Dataset Curation** – Remove synthetic samples from mixed datasets.
|
| 96 |
+
* **Visual Authenticity Verification** – Check the integrity of visual media.
|
| 97 |
+
* **Digital Forensics** – Support image source verification and traceability.
|
|
|
|
|
|