Commit
·
5a1d748
1
Parent(s):
59c47d7
All required Info
Browse files- read_info.txt +36 -0
read_info.txt
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Engagement level detection & Customised Alert System
|
| 2 |
+
|
| 3 |
+
### Detecting engagement using Yolov5 & responding with an alert message using LLM
|
| 4 |
+
|
| 5 |
+
By : [Tanishq Selot](https://github.com/tanishq150802)
|
| 6 |
+
|
| 7 |
+
## Requirements
|
| 8 |
+
* transformers
|
| 9 |
+
* huggingface_hub
|
| 10 |
+
* googletrans==4.0.0-rc1
|
| 11 |
+
* protobuf==3.20
|
| 12 |
+
* torch
|
| 13 |
+
* gradio
|
| 14 |
+
* numpy
|
| 15 |
+
* opencv-python
|
| 16 |
+
* einops
|
| 17 |
+
|
| 18 |
+
All the Yolov5 fine-tuning code is contained within **Yolov5_finetuning.ipynb**. Image examples can be found below.
|
| 19 |
+
|
| 20 |
+
### App flow **run.py**
|
| 21 |
+
|
| 22 |
+
* When the code runs, webcam starts.
|
| 23 |
+
* User can adjust his/her face at the center and press **capture** to capture the image.
|
| 24 |
+
* Yolov5 classifies the image as "looking away", "awake" or "drowsy" & the output is displayed.
|
| 25 |
+
* This response is passed to **https://huggingface.co/microsoft/phi-1_5** LLM endpoint & the alert message is displayed as well.
|
| 26 |
+
|
| 27 |
+
### Example
|
| 28 |
+
|
| 29 |
+
Drowsy | Looking away | Awake
|
| 30 |
+
:-------------------------:|:-------------------------: |:-------------------------:
|
| 31 |
+
 |  | 
|
| 32 |
+
|
| 33 |
+
### References
|
| 34 |
+
|
| 35 |
+
* [LLM](https://huggingface.co/microsoft/phi-1_5)
|
| 36 |
+
* [Manually annotated data used for finetuning Yolov5](https://app.roboflow.com/indian-institute-of-technology-indore-kbon5/engagement_level/2)
|