# HearU

![HearU](https://challengepost-s3-challengepost.netdna-ssl.com/photos/production/software_photos/001/401/027/datas/gallery.jpg)

## Inspiration

Speech is one of the fundamental ways for us to communicate with each other, but many are left out of this channel of communication. Around half a billion people around the world suffer from hearing loss, this is something we wanted to address and help remedy.

Without the ability to hear, many are locked out of professional opportunities to advance their careers especially for jobs requiring leadership and teamwork. Even worse, many have trouble creating fulfilling relationships with others due to this communication hurdle.

Our aim is to make this an issue of the past and allow those living with hearing loss to be able to do everything they wish to do easily.
HearU is the first step in achieving this dream.
![Why it matters](https://challengepost-s3-challengepost.netdna-ssl.com/photos/production/software_photos/001/402/222/datas/gallery.jpg)

## What it does

HearU is an inexpensive and easy-to-use device designed to help those with hearing loss understand the world around them by giving them subtitles for the world.
The device attaches directly to the user’s arm and automatically transcribes any speech heard by it to the user onto the attached screen. The device then adds the transcription on a new line to make sure it is distinct for the user.
A vibration motor is also added to alert the user when a new message is added. The device also has a built-in hazard detection to determine if there are hazards in the area such as fire alarms. HearU also allows the user to store conversations that took place locally on the device for privacy.

Most Importantly, HearU allows the user to interact with ease with others around them.

## How we built it

HearU is based on a Raspberry Pi 4, and uses a display, microphone and a servo motor to operate and interface with the user. A Python script analyzes the data from the microphone, and updates the google cloud API, recording the real-time speech and then displaying the information on the attached screen. Additionally, it uses the pyaudio module to measure the decibels of the sounds surrounding the device, and it is graphically presented to the user. The frontend is then rendered with a python library called pyQT. The entire device is enclosed in a custom-built 3d printed housing unit that is attached to the user’s arm.
This project required a large number of different parts that were not standard as we were trying to minimize size as much as possible but ended up integrating beautifully together.

![hardware](https://challengepost-s3-challengepost.netdna-ssl.com/photos/production/software_photos/001/402/221/datas/gallery.jpg)

## Challenges we ran into

The biggest challenge that we ran into was hardware availability. I needed to scavenge for parts between multiple different sites, which made connecting “interesting” to say the least since we had to use 4 HDMI connectors to get the right angles we needed. Finding a screen that was affordable, that would arrive on time and the right size for the project was not easy.

Trying to fit all the components into something that would fit on an arm was extremely difficult as well and we had to compromise in certain areas such as having the power supply in the pocket for weight and size constraints. We started using a raspberry pi zero w at first but found it was too underpowered for what we wanted to do so we had to switch to the pi4. This meant we had to redesign the enclosure.

We were not able to get adequate time to build the extra features we wanted such as automatic translation since we were waiting on the delivery of the parts, as well as spending a lot of time troubleshooting the pi. We even had the power cut out for one of our developers who had all the frontend code locally saved!

In the end, we are happy with what we were able to create in the time frame given.

## Accomplishments that we're proud of

I’m proud of getting the whole hack to work at all! Like I mentioned above, the hardware was an issue to get integrated and we were under a major time crunch. Plus working with a team remotely for a hardware project was difficult and our goal was very ambitious.
We are very proud that we were able to achieve it!

## What we learned

We learned how to create dense electronic devices that manage space effectively and we now have a better understanding of the iterative engineering design process as we went through three iterations of the project. We also learned how to use google cloud as it was the first time any of us have used it. Furthermore, we learned how to create a good-looking UI using just python and pyqt. Lastly, we learned how to manipulate audio and various audio fingerprinting algorithms to match audio to sounds in the environment.

## What's next for HearU

There are many things that we can do to improve HearU.

We would like to unlock the potential for even more communication by adding automatic language translation to break down all borders.

We can also miniaturize the hardware immensely by using more custom-built electronics, to allow HearU to be as cumbersome and as easy to use as possible.

We would work on Integrating a camera to track users’ hands so that we can convert sign language into speech so that those with hearing impairments can easily communicate with others, even if they don’t know sign language.

![Future Steps](https://challengepost-s3-challengepost.netdna-ssl.com/photos/production/software_photos/001/402/223/datas/gallery.jpg)
## Inspiration

About 0.2 - 2% of the population suffers from deaf-blindness and many of them do not have the necessary resources to afford accessible technology. This inspired us to build a low cost tactile, braille based system that can introduce accessibility into many new situations that was previously not possible.

## What it does

We use 6 servo motors controlled by Arduino that mimic braille style display by raising or lowering levers based upon what character to display. By doing this twice per second, even long sentences can be transmitted to the person. All the person needs to do is put their palm on the device. We believe this method is easier to learn and comprehend as well as way cheaper than refreshable braille displays which usually cost more than $5,000 on an average.

## How we built it

We use Arduino and to send commands, we use PySerial which is a Python Library. To simulate the reader, we have also build a smartbot with it that relays information to the device. For that we have used Google's Dialogflow.
We believe that the production cost of this MVP is less than $25 so this product is commercially viable too.

## Challenges we ran into

It was a huge challenge to get the ports working with Arduino. Even with the code right, pyserial was unable to send commands to Arduino. We later realized after long hours of struggle that the key to get it to work is to give some time to the port to open and initialize. So by adding a wait of two seconds and then sending the command, we finally got it to work.

## Accomplishments that we're proud of

This was our first hardware have to pulling something like that together a lot of fun!

## What we learned

There were a lot of things that were learnt including the Arduino port problem. We learnt a lot about hardware too and how serial ports function. We also learnt about pulses and how sending certain amount of pulses we are able to set the servo to a particular position.

## What's next for AddAbility

We plan to extend this to other businesses by promoting it. Many kiosks and ATMs can be integrated with this device at a very low cost and this would allow even more inclusion in the society. We also plan to reduce the prototype size by using smaller motors and using steppers to move the braille dots up and down. This is believed to further bring the cost down to around $15.
## Inspiration

Our inspiration for Servitium comes from a deep commitment to enhancing the lives of senior citizens by providing a centralized hub for essential services tailored to their unique needs.

## What it does

Servitium serves as a comprehensive service hub specifically designed for senior citizens. It streamlines access to various services, making daily tasks and support more accessible and efficient.

Users are provided with a user-friendly interface for finding services that fit their needs. The application also supports use of natural language to find services (e.g. can enter "I would like to find an affordable plumber" in the home search bar).

Users can also leave reviews for service providers which makes it easier for other users to find the service providers that best suit their needs.

## How we built it

Our platform integrates user-friendly interfaces and secure backend systems to ensure a seamless experience for both users and service providers.

* Front-end: React
* Back-end: Node.js, MongoDB
* Authentication: Auth0
* APIs used: OpenAI, Google Maps, Google Places, Nodemailer, Twilio

## Challenges we ran into

* Integrating OpenAI's API to client-side of the application
* Integrating Nodemailer into the back-end of the application

## Accomplishments that we're proud of

* Integrating AI into our stack - users can use natural language to find services
* Connecting back-end and front-end of our application
* Making use of Auth0 to provide authentication
* Introducing real-time texting and emailing notification feature

## What we learned

* Using third-party APIs
* Full stack development (working on both the server and the client side)
* How to setup and leverage MongoDB

## What's next for Servitium

* Setting up the service provider side of the application since the application currently only supports the client side
* Retrieving more data on the service provider side (e.g. geolocation)