## Inspiration

One of our team members' grandfathers went blind after slipping and hitting his spinal cord, going from a completely independent individual to reliant on others for everything. The lack of options was upsetting, how could a man who was so independent be so severely limited by a small accident. There is current technology out there for blind individuals to navigate their home, however, there is no such technology that allows blind AND frail individuals to do so. With an increasing aging population, Elderlyf is here to be that technology. We hope to help our team member's grandfather and others like him regain his independence by making a tool that is affordable, efficient, and liberating.

## What it does

Ask your Alexa to take you to a room in the house, and Elderlyf will automatically detect which room you're currently in, mapping out a path from your current room to your target room. With vibration disks strategically located underneath the hand rests, Elderlyf gives you haptic feedback to let you know when objects are in your way and in which direction you should turn. With an intelligent turning system, Elderlyf gently helps with turning corners and avoiding obstacles.

## How I built it

With a Jetson Nano and RealSense Cameras, front view obstacles are detected and a map of the possible routes are generated. SLAM localization was also achieved using those technologies. An Alexa and AWS Speech to Text API was used to activate the mapping and navigation algorithms. By using two servo motors that could independently apply a gentle brake to the wheels to aid users when turning and avoiding obstacles. Piezoelectric vibrating disks were also used to provide haptic feedback in which direction to turn and when obstacles are close.

## Challenges I ran into

Mounting the turning assistance system was a HUGE challenge as the setup needed to be extremely stable. We ended up laser-cutting mounting pieces to fix this problem.

## Accomplishments that we're proud of

We're proud of creating a project that is both software and hardware intensive and yet somehow managing to get it finished up and working.

## What I learned

Learned that the RealSense camera really doesn't like working on the Jetson Nano.

## What's next for Elderlyf

Hoping to incorporate a microphone to the walker so that you can ask Alexa to take you to various rooms even though the Alexa may be out of range.
## Inspiration

We were inspired by the impact plants have on battling climate change. We wanted something that not only identifies and gives information about our plants, but also provides an indication about what others think about the plant.

## What it does

You can provide an image of a plant, either by uploading a local image or URL to an image. It takes the plant image and matches it with a species, giving you a similarity score, a scientific name, as well as common names. You can click on it to open a modal that displays more information, including the sentiment analysis of its corresponding Wikipedia page as well as a more detailed description of the plant.

## How we built it

We used an API from [Pl@ntNet](https://identify.plantnet.org/) that utilizes image recognition to identify plants. To upload the image, we needed to provide a link to the image path as a parameter. In order to make this compatible with locally uploaded images, we first saved these into Firebase. Then, we passed the identified species through an npm web scraping library called WikiJS to pull the text content from the Wikipedia page. Finally, we used Google Cloud's Natural Language API to perform sentiment analysis on Wikipedia.

## Challenges we ran into

* Finding sources that we can perform sentiment analysis on our plant
* Being able to upload a local image to be identified, which we resolved using Firebase
* Finding the appropriate API/database for plants
* Connecting frontend with backend

## Accomplishments that we're proud of

* Trying out Firebase and Google Cloud for the first time
* Learning frontier image recognition and NLP softwares
* Integrating API's to gather data
* Our beautiful UI

## What we learned

* How to manage and use the Google Cloud's Natural Language API and the PlantNet API
* There are a lot of libraries and API's that already exist to make our life easier

## What's next for Plantr

* Find ways to get carbon sequestration data about the plant
* Apply sentiment analysis on blog posts about plants to obtain better data
## Inspiration

This generation of technological innovation and human factor design focuses heavily on designing for individuals with disabilities. As such, the inspiration for our project was an application of object detection (Darknet YOLOv3) for visually impaired individuals. This user group in particular has limited visual modality, which the project aims to provide.

## What it does

Our project aims to provide the visually impaired with sufficient awareness of their surroundings to maneuver. We created a head-mounted prototype that provides the user group real-time awareness of their surroundings through haptic feedback. Our smart wearable technology uses a computer-vision ML algorithm (convolutional neural network) to help scan the user’s environment to provide haptic feedback as a warning of a nearby obstacle. These obstacles are further categorized by our algorithm to be dynamic (moving, live objects), as well as static (stationary) objects. For our prototype, we filtered through all the objects detected to focus on the nearest object to provide immediate feedback to the user, as well as providing a stronger or weaker haptic feedback if said object is near or far respectively.

## Process

While our idea is relatively simple in nature, we had no idea going in just how difficult the implementation was.

Our main goal was to meet a minimum deliverable product that was capable of vibrating based on the position, type, and distance of an object. From there, we had extra goals like distance calibration, optimization/performance improvements, and a more complex human interface.

Originally, the processing chip (the one with the neural network preloaded) was intended to be the Huawei Atlas. With the additional design optimizations unique to neural networks, it was perfect for our application. After 5 or so hours of tinkering with no progress however, we realized this would be far too difficult for our project.

We turned to a Raspberry Pi and uploaded Google’s pre-trained image analysis network. To get the necessary IO for the haptic feedback, we also had this hooked up to an Arduino which was connected to a series of haptic motors. This showed much more promise than the Huawei board and we had a functional object identifier in no time. The rest of the night was spent figuring out data transmission to the Arduino board and the corresponding decoding/output.

With only 3 hours to go, we still had to finish debugging and assemble the entire hardware rig.

## Key Takeaways

In the future, we all learned just how important having a minimum deliverable product (MDP) was. Our solution could be executed with varying levels of complexity and we wasted a significant amount of time on unachievable pipe dreams instead of focusing on the important base implementation.

The other key takeaway of this event was to be careful with new technology. Since the Huawei boards were so new and relatively complicated to set up, they were incredibly difficult to use. We did not even use the Huawei Atlas in our final implementation meaning that all our work was not useful to our MDP.

## Possible Improvements

If we had more time, there are a few things we would seek to improve.

First, the biggest improvement would be to get a better processor. Either a Raspberry Pi 4 or a suitable replacement would significantly improve the processing framerate. This would make it possible to provide more robust real-time tracking instead of tracking with significant delays.

Second, we would expand the recognition capabilities of our system. Our current system only filters for a very specific set of objects, particular to an office/workplace environment. Our ideal implementation would be a system applicable to all aspects of daily life. This means more objects that are recognized with higher confidence.

Third, we would add a robust distance measurement tool. The current project uses object width to estimate the distance to an object. This is not always accurate unfortunately and could be improved with minimal effort.