# Pitch

Every time you throw trash in the recycling, you either spoil an entire bin of recyclables, or city workers and multi-million dollar machines separate the trash out for you. We want to create a much more efficient way to sort garbage that also trains people to sort correctly and provides meaningful data on sorting statistics.

Our technology uses image recognition to identify the waste and opens the lid of the correct bin. When the image recognizer does not recognize the item, it opens all bins and trusts the user to deposit it. It also records the number of times a lid has been opened to estimate what and how much is in each bin.

The statistics would have many applications. Since we display the proportion of all garbage in each bin, it will motivate people to compost and recycle more. It will also allow cities to recognize when a bin is full based on how much it has collected, allowing garbage trucks to optimize their routes. In addition, information about what items are commonly thrown into the trash would be useful to material engineers who can design recyclable versions of those products.

Future improvements include improved speed and reliability, IOTA blockchain integration, facial recognition for personalized statistics, and automatic self-learning.

# How it works

1. Raspberry Pi uses webcam and opencv to look for objects
2. When an object is detected the pi sends the image to the server
3. Server sends image to cloud image recognition services (Amazon Rekognition & Microsoft Azure) and determines which bin should be open
4. Server stores information and statistics in a database
5. Raspberry Pi gets response back from server and moves appropriate bin
## Inspiration

Canadians produce more garbage per capita than any other country on earth, with the United States ranking third in the world. In fact, Canadians generate approximately 31 million tonnes of garbage a year. According to the Environmental Protection Agency, 75% of this waste is recyclable. Yet, only 30% of it is recycled. In order to increase this recycling rate and reduce our environmental impact, we were inspired to propose a solution through automating waste sorting.

## What it does

Our vision takes control away from the user, and lets the machine do the thinking when it comes to waste disposal!
By showing our app a type of waste through the webcam, we detect and classify the category of waste into either recyclable, compost, or landfill. From there, the appropriate compartment is opened to ensure that the right waste gets to the right place!

## How we built it

Using TensorFlow and object detection, a python program analyzes the webcam image input and classifies the objects shown. The TensorFlow data is then collected and pushed to our MongoDB Atlas database via Google Cloud. For this project, we used machine learning and used a single shot detector model to maintain a balance between accuracy and speed. For the hardware, an Arduino 101 and a step motor were responsible for manipulating the position of the lid and opening the appropriate compartment.

## Challenges we ran into

We had many issues with training our ML Models on Google Cloud, due to the meager resources provided by Google. Another issue we encountered was finding the right datasets, due to the novelty of our product. Due to these setbacks, we resorted to modifying a TensorFlow provided model.

## Accomplishments that I'm proud of

We managed to work through difficulties and learned a lot during the process! We learned to connect TensorFlow, Arduino, MongoDB, and Express.js to create a synergistic project.

## What's next for Trash Code

In the future, we aim to create a mobile app for improved accessibility and to create a fully customized trained ML model. We also hope to design a fully functional full-sized prototype with the Arduino.
## Inspiration

The beginnings of this idea came from long road trips. When driving having good
visibility is very important. When driving into the sun, the sun visor never seemed
to be able to actually cover the sun. When driving at night, the headlights of
oncoming cars made for a few moments of dangerous low visibility. Why isn't there
a better solution for these things? We decided to see if we could make one, and
discovered a wide range of applications for this technology, going far beyond
simply blocking light.

## What it does

EyeHUD is able to track objects on opposite sides of a transparent LCD screen in order to
render graphics on the screen relative to all of the objects it is tracking. i.e. Depending on where the observer and the object of interest are located on the each side of the screen, the location of the graphical renderings are adjusted

Our basic demonstration is based on our original goal of blocking light. When sitting
in front of the screen, eyeHUD uses facial recognition to track the position of
the users eyes. It also tracks the location of a bright flash light on the opposite side
of the screen with a second camera. It then calculates the exact position to render a dot on the screen
that completely blocks the flash light from the view of the user no matter where
the user moves their head, or where the flash light moves. By tracking both objects
in 3D space it can calculate the line that connects the two objects and then where
that line intersects the monitor to find the exact position it needs to render graphics
for the particular application.

## How we built it

We found an LCD monitor that had a broken backlight. Removing the case and the backlight
from the monitor left us with just the glass and liquid crystal part of the display.
Although this part of the monitor is not completely transparent, a bright light would
shine through it easily. Unfortunately we couldn't source a fully transparent display
but we were able to use what we had lying around. The camera on a laptop and a small webcam
gave us the ability to track objects on both sides of the screen.

On the software side we used OpenCV's haar cascade classifier in python to perform facial recognition.
Once the facial recognition is done we must locate the users eyes in their face in pixel space for
the user camera, and locate the light with the other camera in its own pixel space. We then wrote
an algorithm that was able to translate the two separate pixel spaces into real 3D space, calculate
the line that connects the object and the user, finds the intersection of this line and the monitor,
then finally translates this position into pixel space on the monitor in order to render a dot.

## Challenges we ran Into

First we needed to determine a set of equations that would allow us to translate between the three separate
pixel spaces and real space. It was important not only to be able to calculate this transformation, but
we also needed to be able to calibrate the position and the angular resolution of the cameras. This meant
that when we found our equations we needed to identify the linearly independent parts of the equation to figure
out which parameters actually needed to be calibrated.

Coming up with a calibration procedure was a bit of a challenge. There were a number of calibration parameters
that we needed to constrain by making some measurements. We eventually solved this by having the monitor render
a dot on the screen in a random position. Then the user would move their head until the dot completely blocked the
light on the far side of the monitor. We then had the computer record the positions in pixel space of all three objects.
This then told the computer that these three pixel space points correspond to a straight line in real space.
This provided one data point. We then repeated this process several times (enough to constrain all of the degrees of freedom
 in the system). After we had a number of data points we performed a chi-squared fit to the line defined by these points
 in the multidimensional calibration space. The parameters of the best fit line determined our calibration parameters to use
in the transformation algorithm.

This calibration procedure took us a while to perfect but we were very happy with the speed and accuracy we were able to calibrate at.

Another difficulty was getting accurate tracking on the bright light on the far side of the monitor. The web cam we were
using was cheap and we had almost no access to the settings like aperture and exposure which made it so the light would
easily saturate the CCD in the camera. Because the light was saturating and the camera was trying to adjust its own exposure,
other lights in the room were also saturating the CCD and so even bright spots on the white walls were being tracked as well.
We eventually solved this problem by reusing the radial diffuser that was on the backlight of the monitor we took apart.
This made any bright spots on the walls diffused well under the threshold for tracking. Even after this we had a bit of trouble
locating the exact center of the light as we were still getting a bit of glare from the light on the camera lens. We were
able to solve this problem by applying a gaussian convolution to the raw video before trying any tracking. This allowed us
to accurately locate the center of the light.

## Accomplishments that we are proud of

The fact that our tracking display worked at all we felt was a huge accomplishments. Every stage of this project felt like a
huge victory. We started with a broken LCD monitor and two white boards full of math. Reaching a well working final product
was extremely exciting for all of us.

## What we learned

None of our group had any experience with facial recognition or the OpenCV library. This was a great opportunity to dig into
a part of machine learning that we had not used before and build something fun with it.

## What's next for eyeHUD

Expanding the scope of applicability.

* Infrared detection for pedestrians and wildlife in night time conditions
* Displaying information on objects of interest
* Police information via license plate recognition

Transition to a fully transparent display and more sophisticated cameras.

General optimization of software.