## Inspiration

I've tutored students in math and physics for many years and very early on, it quickly became evident that these students were incredibly bright, but they just never asked for clarification in class. Often in tutoring sessions, I am stuck wondering whether a student is confused or not and oftentimes it hinders the progression of the class. I can only wonder how it must be when it is not just a one on one interaction.

## What it does

1. Automatically takes attendance through facial recognition software using only one image. For deployment purposes, the image could be just a yearbook photo which nearly everyone takes.
2. Scans the emotions and facial expressions of the students in the classroom and tells the teacher when the majority of the students are confused.

## How I built it

For the frontend, a MVVM structure was created with UIKit in swift. The backend was built with python and flask. The facial recognition model came from openCV and the emotion recognition model was the VGG19 model trained on the FER-13 dataset.

## Challenges we ran into

The first main challenge was using flask for the backend because I had never done it before but it quickly became evident that it wasn't very difficult and was quite similar to HTTP requests in JavaScript. The next main challenge was getting both AI algorithms to run in tandem with each other and produce a combined output that could eventually be extracted with the HTTP requests. The final hurdle was creating all the niche features like tracking all the total time the students are in the classroom and warning the teacher about each student that is skipping class.

## Accomplishments that we're proud of

Building a fully functioning, ready to deploy app that can be used in public schools today in under 24 hours.

## What's next for teach.ai

I plan on trying to take this company forward. This is not just some idea I had for only a hackathon, but rather, my vision is much greater. This tool would be extremely useful all accross the country and could cause extreme changes in how our future generations grow and develop their minds. I plan on building a few fully working prototypes and seeing if I can try and deploy them in the schools near my house. From there I can build a reputation and scale the company further.
## Inspiration

**As Computer Science is a learning-intensive discipline, students tend to aspire to their professors**. We were inspired to hack this weekend by our beloved professor Daniel Zingaro (UTM). Answering questions in Dan's classes often ends up being a difficult part of our lectures, as Dan is visually impaired. This means students are expected to yell to get his attention when they have a question, directly interrupting the lecture. Teachers Pet could completely change the way Dan teaches and interacts with his students.

## What it does

Teacher's Pet (TP) empowers students and professors by making it easier to ask and answer questions in class. Our model helps to streamline lectures by allowing professors to efficiently target and destroy difficult and confusing areas in curriculum. Our module consists of an app, a server, and a camera. A professor, teacher, or presenter may download the TP app, and receive a push notification in the form of a discrete vibration whenever a student raises their hand for a question. This eliminates students feeling anxious for keeping their hands up, or professors receiving bad ratings for inadvertently neglecting students while focusing on teaching.

## How we built it

We utilized an Azure cognitive backend and had to manually train our AI model with over 300 images from around UofTHacks. Imagine four sleep-deprived kids running around a hackathon asking participants to "put your hands up". The AI is wrapped in a python interface, and takes input from a camera module. The camera module is hooked up to a Qualcomm dragonboard 410c, which hosts our python program. Upon registering, you may pair your smartphone to your TP device through our app, and set TP up in your classroom within seconds. Upon detecting a raised hand, TP will send a simple vibration to the phone in your pocket, allowing you to quickly answer a student query.

## Challenges we ran into

We had some trouble accurately differentiating when a student was stretching vs. actually raising their hand, so we took a sum of AI-guess-accuracies over 10 frames (250ms). This improved our AI success rate exponentially.

Another challenge we faced was installing the proper OS and drivers onto our Dragonboard. We had to "Learn2Google" all over again (for hours and hours). Luckily, we managed to get our board up and running, and our project was up and running!

## Accomplishments that we're proud of

Gosh darn we stayed up for a helluva long time - longer than any of us had previously. We also drank an absolutely disgusting amount of coffee and red bull. In all seriousness, we all are proud of each others commitment to the team. Nobody went to sleep while someone else was working. Teammates went on snack and coffee runs in freezing weather at 3AM. Smit actually said a curse word. Everyone assisted on every aspect to some degree, and in the end, that fact likely contributed to our completion of TP. The biggest accomplishment that came from this was knowledge of various new APIs, and the gratification that came with building something to help our fellow students and professors.

## What we learned

Among the biggest lessons we took away was that **patience is key**. Over the weekend, we struggled to work with datasets as well as our hardware. Initially, we tried to perfect as much as possible and stressed over what we had left to accomplish in the timeframe of 36 hours. We soon understood, based on words of wisdom from our mentors, that \_ the first prototype of anything is never perfect \_. We made compromises, but made sure not to cut corners. We did what we had to do to build something we (and our peers) would love.

## What's next for Teachers Pet

We want to put this in our own classroom. This week, our team plans to sit with our faculty to discuss the benefits and feasibility of such a solution.
## Inspiration

After years of teaching methods remaining constant, technology has not yet infiltrated the classroom to its full potential. One day in class, it occurred to us that there must be a correlation between students behaviour in classrooms and their level of comprehension.

## What it does

We leveraged Apple's existing API's around facial detection and combined it with the newly added Core ML features to track students emotions based on their facial queues. The app can follow and analyze up to ~ ten students and provide information in real time using our dashboard.

## How we built it

The iOS app integrated Apple's Core ML framework to run a [CNN](https://www.openu.ac.il/home/hassner/projects/cnn_emotions/) to detect people's emotions from facial queues. The model was then used in combination with Apple's Vision API to identify and extract student's face's. This data was then propagated to Firebase for it to be analyzed and displayed on a dashboard in real time.

## Challenges we ran into

Throughout this project, there were several issues regarding how to improve the accuracy of the facial results. Furthermore, there were issues regarding how to properly extract and track users throughout the length of the session. As for the dashboard, we ran into problems around how to display data in real time.

## Accomplishments that we're proud of

We are proud of the fact that we were able to build such a real-time solution. However, we are happy to have met such a great group of people to have worked with.

## What we learned

Ozzie learnt more regarding CoreML and Vision frameworks.

Haider gained more experience with front-end development as well as working on a team.

Nakul gained experience with real-time graphing as well as helped developed the dashboard.

## What's next for Flatline

In the future, Flatline could grow it's dashboard features to provide more insight for the teachers. Also, the accuracy of the results could be improved by training a model to detect emotions that are more closely related to learning and student's behaviours.