## Inspiration

We wanted to learn about machine learning. There are thousands of sliding doors made by Black & Decker and they're all capable of sending data about the door. With this much data, the natural thing to consider is a machine learning algorithm that can figure out ahead of time when a door is broken, and how it can be fixed. This way, we can use an app to send a technician a notification when a door is predicted to be broken. Since technicians are very expensive for large corporations, something like this can save a lot of time, and money that would otherwise be spent with the technician figuring out if a door is broken, and what's wrong with it.

## What it does

DoorHero takes attributes (eg. motor speed) from sliding doors and determines if there is a problem with the door. If it detects a problem, DoorHero will suggest a fix for the problem.

## How we built it

DoorHero uses a Tensorflow Classification Neural Network to determine fixes for doors. Since we didn't have actual sliding doors at the hackathon, we simulated data and fixes. For example, we'd assign high motor speed to one row of data, and label it as a door with a problem with the motor, or we'd assign normal attributes for a row of data and label it as a working door.

The server is built using Flask and runs on [Floydhub](https://floydhub.com). It has a Tensorflow Neural Network that was trained with the simulated data. The data is simulated in an Android app. The app generates the mock data, then sends it to the server. The server evaluates the data based on what it was trained with, adds the new data to its logs and training data, then responds with the fix it has predicted.

The android app takes the response, and displays it, along with the mock data it sent.

In short, an Android app simulates the opening and closing of a door and generates mock data about the door, which is sends everytime the door "opens", to a server using a Flask REST API. The server has a trained Tensorflow Neural Network, which evaluates the data and responds with either "No Problems" if it finds the data to be normal, or a fix suggestion if it finds that the door has an issue with it.

## Challenges we ran into

The hardest parts were:

* Simulating data (with no background in sliding doors, the concept of sliding doors sending data was pretty abstract).
* Learning how to use machine learning (turns out this isn't so easy) and implement tensorflow
* Running tensorflow on a live server.

## Accomplishments that we're proud of

## What we learned

* A lot about modern day sliding doors
* The basics of machine learning with tensorflow
* Discovered floydhub

## What we could have improve on

There are several things we could've done (and wanted to do) but either didn't have time or didn't have enough data to. ie:

* Instead of predicting a fix and returning it, the server can predict a set of potential fixes in order of likelihood, then send them to the technician who can look into each suggestion, and select the suggestion that worked. This way, the neural network could've learned a lot faster over time. (Currently, it adds the predicted fix to its training data, which would make for bad results
* Instead of having a fixed set of door "problems" for the door, we could have built the app so that in the beginning, when the neural network hasn't learned yet, it asks the technician for input after everytime the fix the door (So it can learn without data we simulated as this is what would have to happen in the normal environment)
* We could have made a much better interface for the app
* We could have added support for a wider variety of doors (eg. different models of sliding doors)
* We could have had a more secure (encrypted) data transfer method
* We could have had a larger set of attributes for the door
* We could have factored more into decisions (for example, detecting a problem if a door opens, but never closes).
## Inspiration

The biggest weakness modern is AI doesn't have any context about our daily lives. With our GlimpseAI glasses, we give AI the power to use audio and visual cues learn and understand from the world around us and provide customized help to enrich our lives.

## What it does

GlimpseAI glasses use a powerful combination of audio input paired with TogetherAI's robust LLMs and visual input put together a with custom object classification computer vision model to comprehend what goes on in the user's life. Later, when the user has a question about something that happened earlier (like whether they took their pills, or where they put their keys), the AI can look back through the day and help the user answer their question. GlimpseAI also helps ease the user's life by giving the ability to perform a series of different actions for the user. For example, GlimpseAI's integration with Google Calendar allows it to schedule meetings whenever it recognizes the user wants to schedule one, and its integration with Slack allows it to send out important messages just by vocalizing the thought.

## How we built it

For the backend, we used a Flask router to interact with the audio input. It transcribes the user's voice to text using Google Cloud's speech to text model, and then leverages the LLMs from TogetherAI to contextualize what's going on. The backend uses Selenium to integrate with common technologies like Google Calendar and Slack. To give a visualized summary of what the AI is understanding and contextualizing, we also have a Next.js/React frontend using Axios to send requests to the Flask backend. Finally, the vision model uses Pytorch to understand the world in front of us.

## Challenges we ran into

There were several challenges with integrating software like Slack and Google Calendar. It was also difficult to train the Computer Vision model. We had a lot of difficulty with the hardware equipment here, however we overcame a lot of the difficulties by coming up with workarounds such as using our laptop mic & attaching an external webcam to create a prototype.

## Accomplishments that we're proud of

We're proud of our ability to improvise & change things on the spot to make our product better. We pivoted a lot throughout the hackathon & despite the slow wifi speeds and lack of hardware equipment, we made do with what we had and persevered which we are really proud of.

## What's next for GlimpseAI

Next, we will integrate more actions and integrate a more robust vision model to understand more common objects and things. We would create a developer sdk so other people could build more actions upon our work in an app store-like marketplace. Finally, we would try to bring more offline features into the glasses and create a prototype with better hardware so that the glasses could be worn in an everyday setting.
## Inspiration

We were interested in developing a solution to automate the analysis of microscopic material images.

## What it does

Our program utilizes image recognition and image processing tools such as edge detection, gradient analysis, gaussian/median/average filters, morphologies, image blending etc. to determine specific shapes of a microscopic image and apply binary thresholds for analysis. In addition, the program has the ability to differentiate between light and dark materials under poor lighting conditions, as well as calculate the average surface areas of grains and the percentage of dark grains.

## How we built it

We used Python algorithms incorporated with OpenCV tools in the PyCharm developing environment.

## Challenges we ran into

Contouring for images was extremely difficult considering there were many limitations and cleaning/calibrating data and threshold values. The time constraints also impacted us, as we would have liked to be able to develop a more accurate algorithm for our image analysis software.

## Accomplishments that we're proud of

Making a breakthrough in iterative masking of the images to achieve an error percentage consistently below 0.5%. We're also incredibly proud of the fact that we were able to complete the majority of the challenge tasks as well as develop a user-friendly interface.

## What we learned

We became better equipped with Python and the opensource materials available to all of us. We also learned valuable computer vision skills through practical applications as well as a developed a better understanding of data processing algorithms.

## What's next for Material Arts 2000

We're looking to further refine our algorithms so that it will be of more practical use in the future. Potentially looking to expand from the specific field of microscopic materials to develop a more widely applicable algorithm.