## Inspiration

There are over 1.3 billion people in the world who live with some form of vision impairment. Often, retrieving small objects, especially off the ground, can be tedious for those without complete seeing ability. We wanted to create a solution for those people where technology can not only advise them, but physically guide their muscles in their daily life interacting with the world.

## What it does

ForeSight was meant to be as intuitive as possible in assisting people with their daily lives. This means tapping into people's sense of touch and guiding their muscles without the user having to think about it. ForeSight straps on the user's forearm and detects objects nearby. If the user begins to reach the object to grab it, ForeSight emits multidimensional vibrations in the armband which guide the muscles to move in the direction of the object to grab it without the user seeing its exact location.

## How we built it

This project involved multiple different disciplines and leveraged our entire team's past experience. We used a Logitech C615 camera and ran two different deep learning algorithms, specifically convolutional neural networks, to detect the object. One CNN was using the Tensorflow platform and served as our offline solution. Our other object detection algorithm uses AWS Sagemaker recorded significantly better results, but only works with an Internet connection. Thus, we use a two-sided approach where we used Tensorflow if no or weak connection was available and AWS Sagemaker if there was a suitable connection. The object detection and processing component can be done on any computer; specifically, a single-board computer like the NVIDIA Jetson Nano is a great choice. From there, we powered an ESP32 that drove the 14 different vibration motors that provided the haptic feedback in the armband. To supply power to the motors, we used transistor arrays to use power from an external Lithium-Ion battery. From a software side, we implemented an algorithm that accurately selected and set the right strength level of all the vibration motors. We used an approach that calculates the angular difference between the center of the object and the center of the frame as well as the distance between them to calculate the given vibration motors' strength. We also built a piece of simulation software that draws a circular histogram and graphs the usage of each vibration motor at any given time.

## Challenges we ran into

One of the major challenges we ran into was the capability of Deep Learning algorithms on the market. We had the impression that CNN could work like a “black box” and have nearly-perfect accuracy. However, this is not the case, and we experienced several glitches and inaccuracies. It then became our job to prevent these glitches from reaching the user’s experience.

Another challenge we ran into was fitting all of the hardware onto an armband without overwhelming the user. Especially on a body part as used as an arm, users prioritize movement and lack of weight on their devices. Therefore, we aimed to provide a device that is light and small.

## Accomplishments that we're proud of

We’re very proud that we were able to create a project that solves a true problem that a large population faces. In addition, we're proud that the project works and can't wait to take it further!

Specifically, we're particularly happy with the user experience of the project. The vibration motors work very well for influencing movement in the arms without involving too much thought or effort from the user.

## What we learned

We all learned how to implement a project that has mechanical, electrical, and software components and how to pack it seamlessly into one product.

From a more technical side, we gained more experience with Tensorflow and AWS. Also, working with various single board computers taught us a lot about how to use these in our projects.

## What's next for ForeSight

We’re looking forward to building our version 2 by ironing out some bugs and making the mechanical design more approachable. In addition, we’re looking at new features like facial recognition and voice control.
## Team

Sam Blouir

Pat Baptiste

## Inspiration

Seeing the challenges of blind people navigating, especially in new areas and being inspired by someone who wanted to remain blind anonymously

## What it does

Uses computer vision to find nearby objects in the vicinity of the wearer and sends information (X,Y coordinates, depth, and size) to the wearer in real-time so they can enjoy walking

## How I built it

Python, C, C++, OpenCV, and an Arduino

## Challenges I ran into

Performance and creating disparity maps from stereo cameras with significantly different video output qualities

## Accomplishments that I'm proud of

It works!

## What I learned

We learned lots about using Python, OpenCV, an Arduino, and integrating all of these to create a hardware hack

## What's next for Good Vibrations

Better depth-sensing and miniaturization!
## Inspiration

With the COVID-19 crisis, it has become more difficult to connect with others so we thought of having music being a simple icebreaker in conversations.

## What it does

Muse is a website that combines music streaming and social networking to allow users to meet each other with similar musical interests. Muse will match users on the platform who both love the same song. Users can then be navigated to the direct messaging page on Muse where they can discuss more their interests and hobbies separately. There is also a community page where everyone can share their music preferences.

## How WE built it

React/Firebase/HTML/CSS

## Challenges WE ran into

We were very lost on how to approach the Spotify API and ultimately had to switch our project approach towards the end.

## Accomplishments that We're proud of

This is the first full web development project that we have completed and though it's not perfect, we learned a lot.

## What's next for Muse

More implementation of the Spotify API to gather their music activities to better match users.