## Inspiration

Road safety has become an escalating concern in recent years. According to Transport Canada, the number of collisions and causalities has been rising in the past three years. Meanwhile, as AI technology grows exponentially, we identified a niche where we could leverage powerful AI to offer valuable advice and feedback to drivers of all levels, effectively promoting road safety.

## What it does

Our system utilizes Computer Vision (CV) techniques and gyroscopes to collect meaningful data about driving performance, such as whether one does shoulder checks on turns or drives when drowsy. The data is then passed to the backend and analyzed by a Large Language Model (LLM). When users want to review their performance and ask for advice, they can ask the LLM to find relevant driving records and offer helpful insights.

Examples of potential use cases are supplementing driving lessons or exams for learners, encouraging concerned drivers to enforce good driving habits, providing objective evaluation of professional driving service quality, etc.

## How we built it

Our system consists of

**a data collection script (Python, React Native),**

* Runs a CV algorithm (utilizing roboflow models) and streams the output with the video and gyroscope (from mobile) data to the frontend web app

**a frontend web app (React),**

* Receives and displays driving performance data
* Provides an interface to review the driving records and interact with the LLM to get valuable insights
Authenticates user logins with Auth0

**a backend (Flask),**

* Connects to Google Gemini for LLM interactions
* Transfers data and LLM outputs between the frontend and the database (Utilizing VectorSearch to extract relevant trip records as context for LLM to generate advice)

**a database (MongoDB Atlas),**

* Saves and transfers metadata and analysis information of each driving trip
* Configured to support VectorSearch

**a cloud storage service (Google Cloud Storage)**

* Hosts driving videos which are media data of larger sizes

## Challenges we ran into

* Setting up web sockets to connect individual components for real-time data transfer
* Configuring Auth0 to perform authentication correctly within the React app
* Deciding how to store videos (Saving them to DB as BLOB vs Using paid cloud storage service)

## Accomplishments that we're proud of

* Built components that achieve their corrseponding functionalities (Idenify shoulder checks and closed eyes, interacting with LLM, querying database by vectors, etc.)
* Overcame or worked around errors arisen from using libraries or SDKs

## What we learned

* Collaborating as a small team to deliver quickly
* Utilizing web sockets for real-time data transfer
* Utilizing vectorSearch in Mongodb Atlas to query documents
* Utilizing Auth0 for authentication
* Connecting various programs and technologies to construct the end-to-end point

## What's next for DriveInsight

* Consulting domain experts (drivers, driving examiners, etc.) for more driving habits to monitor
* Fine-tuning specialized LLMs for even more robust and insightful responses
## Inspiration

We were looking for an innovative solution to keep us aware of what we were eating.

## What it does

Nutlogger is a web and mobile application that tracks nutritional data over a period of the day, month, year. With this data, we hope that users can get a better understanding of their eating habits.

## How I built it

Lots of hard work and patience. The web application was built with MERN and the mobile application was built with Android Studio.

## Challenges I ran into

Parsing the information for Google's vision API was difficult.

## Accomplishments that I'm proud of

Developing a functional application that actually works

## What I learned

* Google cloud platform
* React Typescript
* android camera

## What's next for Nutlogger

* account system and profiles
* admin panel for doctors
* chat with nutritionists

## Credits:

Icons made by Freepik from [www.flaticon.com](http://www.flaticon.com) is licensed by CC 3.0 BY
## Inspiration

Ontario Provincial Police say distracted driving continues to be the No. 1 cause of accidents in the province. According to AAA, distracted driving was a factor in nearly 6 out of 10 moderate-to-severe teen crashes. This personal element was a contributing factor in our decision to pursue this project. Through DriveToArrive, we hope to reduce the number of crashes as a result of distracted driving and encourage safe driving practices.

## What it does

A video stream is taken in from the camera. The video frames are continually analyzed to detect the presence or absence of an individual’s face and eyes. If a face or eyes are unable to be detected, an audible message is played alerting the driver to pay attention to the road.

## How we built it

DriveToArrive implements Python and OpenCV. A multitude of facial image data is provided to the program. This data is comprised of images that contain arbitrarily good or bad images of a face. Incoming frames from a video stream are analyzed using this pre-trained model, and the logic-based decision-making process is executed.

When an individual frame is analyzed if a face or eyes is detected a 1 is stored within a continually updated array. If not detected, a zero is stored. This array is used to calculate a rolling percentage of consecutive frames in which the driver is distracted. 75% was chosen as the threshold for determining whether a driver is distracted. The driver is alerted with a different audible message depending on if the program determines an absence of a face or the presence of a face but the absence of eyes.

## Challenges we ran into

Initially, we had planned to use a Raspberry Pi to run our application, but when borrowing hardware a critical cable was missing. This held up the progress for a long period of time as the cable had to be purchased to progress. The other challenge was getting access to a monitor as a limited number were available at the facility, all of which were on high demand. As we were trying to resolve these issues we lost a great deal of time, which limited the time invested into the project.

Additionally, we were going to use IR sensors to detect, more accurately, if a persons’ eyes are opened or closed while driving, but the IR sensors did not behave as expected. This once again resulted in a great loss of time as implementing and testing the sensors was time-consuming and did not end up working.
While using OpenCV, our team was having difficulties distinguishing between if a driver had their eyes closed or face turned away. This needed to be handled by OpenCV after the IR sensors were not an option in order to allow us to detect the eyes’ position.

## Accomplishments that we're proud of

As a group, we are very proud of the accuracy and speed at which the script can identify the lack of face and eyes in the image stream.

We are also proud of our resilience when faced with the challenges provided by the usage of the Raspberry Pi. As described in the previous section, we ran into multiple issues with the Raspberry Pi including not having the proper equipment to utilize the versatility that it provides.

## What we learned

Most of the team had never programmed in Python before, leading to skill development.

OpenCV was a library that none of us knew before attending this hackathon, so this was a new skill all of us learned as well.

## What's next for Drive2Arrive

In the future, we are looking to continue development on DriveToArrive, extending the service to a mobile application to increase the scalability of the project.