File size: 5,524 Bytes
78cfa8f f0a6185 78cfa8f f0a6185 78cfa8f f0a6185 78cfa8f f0a6185 78cfa8f b197266 b2890ad 78cfa8f f0a6185 78cfa8f f0a6185 b2890ad b197266 b2890ad f0a6185 b2890ad b197266 78cfa8f 1f2ead6 2c6ecab 1f2ead6 78cfa8f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
---
tags:
- text-to-image
- lora
- diffusers
- template:diffusion-lora
widget:
- text: >-
Person with expression of pain due to a heart attack, Elderly man at a
sports stadium surrounded by a crowd, clutching his chest with a distressed
look, indicating a heart attack.
parameters:
negative_prompt: >-
blurry, deformed face, bad anatomy, poorly drawn face, out of focus, ugly,
noisy, extra fingers, distorted, grainy, worst quality, low quality, low
resolution, illustration, dull, watermark, close-up, 3d, 2d, painting,
sketch, render, cartoon, grain, kitsch
output:
url: images/sd-2.1-infarct-lora-084-766027158.jpg
- text: >-
Person with expression of pain due to a heart attack, Middle-aged woman in
her kitchen leaning on a counter, struggling to breathe, showing signs of a
cardiac emergency.
parameters:
negative_prompt: >-
blurry, deformed face, bad anatomy, poorly drawn face, out of focus, ugly,
noisy, extra fingers, distorted, grainy, worst quality, low quality, low
resolution, illustration, dull, watermark, close-up, 3d, 2d, painting,
sketch, render, cartoon, grain, kitsch
output:
url: images/sd-2.1-infarct-lora-019-623607189.jpg
base_model: stabilityai/stable-diffusion-2-1-base
instance_prompt: Person with expression of pain due to a heart attack, infarct
license: mit
---
# Infarct Image
<Gallery />
## Model description
# InfarctImage - LoRA Fine-Tuned Model for Heart Attack Simulation
## 📌 Description
**InfarctImage** is a LoRA-based model fine-tuned on **Stable Diffusion 2.1**, designed to generate realistic images of people simulating a heart attack. This model was developed as part of a study on synthetic dataset generation for human activity recognition and medical emergency monitoring applications.
🔗 **Related Article:** Coming soon.
## 🎯 Objective
The model addresses the issue of data scarcity in medical and anomaly detection environments. Generating high-quality synthetic images enables:
- Expanding existing datasets without relying on real-world data.
- Overcoming ethical and logistical restrictions in medical image collection.
- Enhancing AI-based detection of critical events like heart attacks.
## 📥 Download and Installation
To use this model with **Diffusers**, follow these steps:
```python
from diffusers import DiffusionPipeline
import torch
# Load the base model
infarct_pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1-base", torch_dtype=torch.float16, use_safetensors=True)
# Load LoRA weights
infarct_pipe.load_lora_weights("Gavit0/InfarctImage")
# Move to GPU if available
infarct_pipe.to(torch.device("cuda" if torch.cuda.is_available() else "cpu"))
print("Model loaded successfully!")
```
## 🏆 Usage Examples
You can generate images using prompts like:
```python
prompt = ("Elderly man at a sports stadium surrounded by a crowd, "
"clutching his chest with a distressed look, indicating a heart attack."
)
negative_prompt = (
"blurry, deformed face, bad anatomy, poorly drawn face, out of focus, ugly, noisy, extra fingers, "
"distorted, grainy, worst quality, low quality, low resolution, illustration, "
"dull, watermark, close-up, 3d, 2d, painting, sketch, render, cartoon, grain, kitsch"
)
trigger = "Person with expression of pain due to a heart attack, "
full_prompt = f"{trigger}, {prompt}"
image = infarct_pipe(prompt=full_prompt, negative_prompt=negative_prompt,
guidance_scale=4, num_inference_steps=40).images[0]
image.show()
```
Full examples in:
- 🔗 **[Notebook in GitHub](https://github.com/Turing-IA-IHC/InfarctImage/blob/main/notebooks/infarctImageDemo.ipynb)**
- [](https://colab.research.google.com/github/Turing-IA-IHC/InfarctImage/blob/main/notebooks/infarctImageDemo.ipynb)
## 📊 Training Data
The model was trained on a dataset of 100 manually annotated images, including:
- 50 images of people simulating heart attack symptoms.
- 50 images of people in neutral contexts.
The dataset was processed and annotated using **BLIP (Bootstrapping Language-Image Pretraining)** to enhance image descriptions and improve training prompts.
## 🔧 Hyperparameters and Configuration
- **Base Model:** Stable Diffusion 2.1
- **Fine-Tuning Technique:** LoRA (Low-Rank Adaptation)
- **Learning Rate:** 0.0001
- **Batch Size:** 1
- **Epochs:** 10
- **Hardware:** NVIDIA RTX 4090 (24GB VRAM)
## 📈 Model Evaluation
**LPIPS (Learned Perceptual Image Patch Similarity)** was used to evaluate the quality of the generated images. Results show that LoRA fine-tuning improves the perceptual similarity of generated images compared to real training data.
| Model | LPIPS (↓ Better) |
|--------|---------------|
| SD 2.1 Base | 0.7366 |
| SD 2.1 + LoRA | 0.6919 |
## 📜 License
This model is distributed under the **MIT License**.
## Trigger words
You should use `Person with expression of pain due to a heart attack` to trigger the image generation.
You should use `infarct` to trigger the image generation.
## Download model
Weights for this model are available in Safetensors format.
[Download](/Gavit0/InfarctImage/tree/main) them in the Files & versions tab.
|