## Inspiration

Inspired by the laborious...glorious…notorious…effort…that hackers end up expending throughout their time at Hack the North. By which we mean: staying up really late, losing all cognizant ability, and living to regret it. Which is to say: *what we did while building this.*

On a more serious note: sleep deprivation and fatigue have serious effects on both mental and physical health. Did you know some of the *worst workplace accidents in history*, spanning multiple industries, can be traced back to sleep-deprived workers, night shifts, and human error? As this is an issue that affects a large portion of the population and us on a personal level, especially this weekend, we decided to apply our interest in Adhawk’s eye-tracking technology to this field.

## What it does

Zone uses the data stream output from the Mindlink along with the events it tracks (blink actions/durations, pupil dilation, saccades, etc.) to build a baseline of a worker’s “expected” behaviours. Workers on a shift will be continuously monitored via the Mindlink and the data will be periodically summarized for analysis, in which the Zone software will monitor research-determined attributes to determine whether or not the worker’s level of fatigue permits them to continue working. If Zone finds that the user is too sleepy to safely continue working, it notifies their supervisor via text message that a swap-out is necessary.

## How we built it

First, we modified a pre-existing program written by the Adhawk developer team with basic features that grants us access to a constant stream of relevant data.

Then, we built an analysis model that compares the average of the rate of change in the data of the current time window to the previous window to estimate the general trend of the user’s level of alertness.

Then, we built an analysis model that compares the average of the rate of change for each attribute between “windows”: 15-minute periods, across which the aforementioned average ROC is determined through all data points collected through set 30-second intervals. The purpose of this data is to estimate the general trend of the user’s level of alertness within that period. If the trend changes significantly from previosly logged trends, the analysis will determine that the user is not fit to continue working in their current state.

As for the materials used:
The star of the show was the AdHawk Mindlink, without which we wouldn’t be able to track these metrics;
The GUI was built using Python Tkinter, while the event and stream handling was written in Adhawk’s Python SDK. Finally, we sent the text message via a call to the Twilio SMS API.

## Challenges we ran into

Since our team has never worked with eye-tracking technology before, we had a lot of issues gauging the specific data we wanted from the Mindlink. We also had issues integrating our front end and back end as they were developed independently without forethought of the application structure as a whole. Luckily, creativity fueled by sleep deprivation has gotten us past those hurdles with *maximum frustration!*

## Accomplishments that we're proud of

Our team really stepped out of our comfort zone to work with technology that we’ve never had experience with before. Coming out of two almost all-nighters with a working product that uses the Mindlink’s core features in an innovative solution is a feat we are all extremely proud of. Aside from the project, we feel that the real accomplishment was ultimately the swag we collected along the way.

## What we learned

We learned a lot about eye-tracking technology, application structure and design, as well as Python GUI via Tkinter. Also, virtual environments. *Lots of virtual environments.*

## What's next for zone

As of now, only one of our five collected metrics has a proper analysis method that is functional. Although we have coded analysis algorithms for all metrics, integrating them proved to be an issue that we would aim to resolve in the future to provide accurate data on the user’s level of fatigue. We also hope to set up a database to store user information to provide better comparison to previously logged information for a more accurate and tailored experience with the device.
## Inspiration

We live in a society where data is extremely important. Technology all around us is constantly collecting it and it's generally being used to improve all our lives. Data allows us to make objectively better decisions guided by information. One relatively new way to gather data is through eye-tracking. Eye-tracking has many possible applications, including determining what a user is focusing on. Based on our experience with web development, we understand how tedious it is to get data on how your user is using your website. For example, are they focusing on what we want them to? Or are they spending time getting distracted by other content on the page? Up until now there hasn't been a great way to get this info, but now there is.

EYEs.py is an innovative solution to collect data through tracking eye movements. Compared to alternative methods of data collection, this approach allows developers to get data on user behaviour by tracking how they do things, rather than just what they do. This new analysis method is referred to by industry as behavioural analytics and is growing rapidly. Specifically, EYEs.py takes where the user is looking and creates a heat map from it to determine what the major areas of focus are on the webpage.

## What It Does

The AdHawk MiniLink's eye tracking sensors track the position of the eye and send it through the tethered USB connection to the computer to AdHawk Backend. From here the data travels through the AdHawk Python SDK and into our Python code where we populate a matrix with values from the eyesight tracking. These values are then used for the heat map with the darker areas having a higher amount of time spent looking at them relative to the light areas. To only capture eye movements on the screen, the forward facing camera on the glasses tracks the 4 markers on the corners of the page. Utilizing this allows the glasses to determine when the screens is being looked at.

## How We Built It

We built this project utilizing the AdHawk suite of products:

* AdHawk Microsystems MindLink glasses
* Utilized the AdHawkAPI to enable data collection
* AdHawk Python Software Development Kit (SDK)
* Object-oriented Python code

## Challenges We Ran Into

* Initially, we struggled to collect and store the data from the glasses through the API provided by AdHawk
* We had many issues with working with QT (the GUI system) that was already being used by Adhawk regarding formatting
* We had issues connecting to CockroachDB and sending data over

## Accomplishments That We're Proud Of

Most of us our team is first-time hackers, and we felt proud to complete a MVP (Minimum Viable Product) of an idea that we were all interested in building. It was also super cool to work with the AdHawk MindLink glasses as this technology was truly cutting edge and a rare opportunity. More importantly, we are proud of what we were able to accomplish with a completely foreign technology in roughly 48 hours.

## What We Learned

This project taught us many valuable lessons including:

* Organization (keeping track of everyone's tasks and their status)
* Collaboration (how to effectively utilize Git to work together efficiently)
* Dedication (persevering through a lack of sleep)

Additionally, it helped us improve our coding skills too especially with technologies such as:

* Python3
* CockroachDB (PostGreSQL)
* Github

## What's Next for EYEs.py

One of our favourite aspects of this projects is its potential. There are many features that we believe could be added to improve the usability, versatility, and efficacy, like:

* Create a recording tool to display the order in which each part of the website was viewed (i.e. the user first viewed the top left part of the page)
* Implementing a web-based front end with an API communication layer with Python backend
* Adapting our heat maps to adjust with scrolling and changing routes in a website
* Creating a dashboard for managing different user recordings and grouping the data
* Rescaling to prevent the tracking markers from covering content
* Combine insights with Microsoft Clarity and Google Analytics to get even more data on website performance for a wider variety of scenarios
* Train an AI/machine learning model to suggest changes by noticing pain points in the UI
## Inspiration

In a lot of mass shootings, there is a significant delay from the time at which police arrive at the scene, and the time at which the police engage the shooter. They often have difficulty determining the number of shooters and their location. ViGCam fixes this problem.

## What it does

ViGCam spots and tracks weapons as they move through buildings. It uses existing camera infrastructure, location tags and Google Vision to recognize weapons. The information is displayed on an app which alerts users to threat location.

Our system could also be used to identify wounded people after an emergency incident, such as an earthquake.

## How we built it

We used Raspberry Pi and Pi Cameras to simulate an existing camera infrastructure. Each individual Pi runs a Python script where all images taken from the cameras are then sent to our Django server. Then, the images are sent directly to Google Vision API and return a list of classifications. All the data collected from the Raspberry Pis can be visualized on our React app.

## Challenges we ran into

SSH connection does not work on the HackMIT network and because of this, our current setup involves turning one camera on before activating the second. In a real world situation, we would be using an existing camera network, and not our raspberry pi cameras to collect video data.

We also have had a difficult time getting consistent identification of our objects as weapons. This is largely because, for obvious reasons, we cannot bring in actual weapons. Up close however, we have consistent identification of team member items.

Using our current server set up, we consistently get server overload errors. So we have an extended delay between each image send. Given time, we would implement an actual camera network, and also modify our system so that it would perform object recognition on videos as opposed to basic pictures. This would improve our accuracy. Web sockets can be used to display the data collected in real time.

## Accomplishments that we’re proud of

1) It works!!! (We successfully completed our project in 24 hours.)
2) We learned to use Google Cloud API.
3) We also learned how to use raspberry pi. Prior to this, none on our team had any hardware experience.

## What we learned

1) We learned about coding in a real world environment
2) We learned about working on a team.

## What's next for ViGCam

We are planning on working through our kinks and adding video analysis. We could add sound detection for gunshots to detect emergent situations more accurately. We could also use more machine learning models to predict where the threat is going and distinguish between threats and police officers. The system can be made more robust by causing the app to update in real time. Finally, we would add the ability to use law enforcement emergency alert infrastructure to alert people in the area of shooter location in real time. If we are successful in these aspects, we are hoping to either start a company, or sell our idea.