|
--- |
|
base_model: |
|
- Qwen/Qwen2-VL-2B-Instruct |
|
datasets: |
|
- Tami3/HazardQA |
|
language: |
|
- en |
|
library_name: transformers |
|
license: apache-2.0 |
|
model_name: HazardNet-unsloth-v0.4 |
|
pipeline_tag: image-text-to-text |
|
tags: |
|
- trl |
|
- sft |
|
--- |
|
|
|
# Model Card for HazardNet-unsloth-v0.4 |
|
|
|
This model is a fine-tuned version of [Qwen/Qwen2-VL-2B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct). |
|
It has been trained using [TRL](https://github.com/huggingface/trl). |
|
|
|
The model was presented in the paper [](https://hf.co/papers/2502.20572). |
|
|
|
## Quick start |
|
|
|
```python |
|
from transformers import pipeline |
|
from PIL import Image |
|
import requests |
|
from io import BytesIO |
|
|
|
# Initialize the Visual Question Answering pipeline with HazardNet |
|
hazard_vqa = pipeline( |
|
"image-text-to-text", |
|
model="Tami3/HazardNet" |
|
) |
|
|
|
# Function to load image from a local path or URL |
|
def load_image(image_path=None, image_url=None): |
|
if image_path: |
|
return Image.open(image_path).convert("RGB") |
|
elif image_url: |
|
response = requests.get(image_url) |
|
response.raise_for_status() # Ensure the request was successful |
|
return Image.open(BytesIO(response.content)).convert("RGB") |
|
else: |
|
raise ValueError("Provide either image_path or image_url.") |
|
|
|
# Example 1: Loading image from a local file |
|
try: |
|
image_path = "path_to_your_ego_car_image.jpg" # Replace with your local image path |
|
image = load_image(image_path=image_path) |
|
except Exception as e: |
|
print(f"Error loading image from path: {e}") |
|
# Optionally, handle the error or exit |
|
|
|
# Example 2: Loading image from a URL |
|
# try: |
|
# image_url = "https://example.com/path_to_image.jpg" # Replace with your image URL |
|
# image = load_image(image_url=image_url) |
|
# except Exception as e: |
|
# print(f"Error loading image from URL: {e}") |
|
# # Optionally, handle the error or exit |
|
|
|
# Define your question about potential hazards |
|
question = "Is there a pedestrian crossing the road ahead?" |
|
|
|
# Get the answer from the HazardNet pipeline |
|
try: |
|
result = hazard_vqa(question=question, image=image) |
|
answer = result.get('answer', 'No answer provided.') |
|
score = result.get('score', 0.0) |
|
|
|
print("Question:", question) |
|
print("Answer:", answer) |
|
print("Confidence Score:", score) |
|
except Exception as e: |
|
print(f"Error during inference: {e}") |
|
# Optionally, handle the error or exit |
|
``` |
|
|
|
## Training procedure |
|
|
|
|
|
|
|
This model was trained with SFT. |
|
|
|
### Framework versions |
|
|
|
- TRL: 0.13.0 |
|
- Transformers: 4.47.1 |
|
- Pytorch: 2.5.1+cu121 |
|
- Datasets: 3.2.0 |
|
- Tokenizers: 0.21.0 |
|
|
|
## Citations |
|
|
|
Cite TRL as: |
|
|
|
```bibtex |
|
@misc{vonwerra2022trl, |
|
title = {{TRL: Transformer Reinforcement Learning}}, |
|
author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec}, |
|
year = 2020, |
|
journal = {GitHub repository}, |
|
publisher = {GitHub}, |
|
howpublished = {\url{https://github.com/huggingface/trl}} |
|
} |
|
``` |