## Inspiration

We wanted to tackle a problem that impacts a large demographic of people. After research, we learned that 1 in 10 people suffer from dyslexia and 5-20% of people suffer from dysgraphia. These neurological disorders go undiagnosed or misdiagnosed often leading to these individuals constantly struggling to read and write which is an integral part of your education. With such learning disabilities, learning a new language would be quite frustrating and filled with struggles. Thus, we decided to create an application like Duolingo that helps make the learning process easier and more catered toward individuals.

## What it does

ReadRight offers interactive language lessons but with a unique twist. It reads out the prompt to the user as opposed to it being displayed on the screen for the user to read themselves and process. Then once the user repeats the word or phrase, the application processes their pronunciation with the use of AI and gives them a score for their accuracy. This way individuals with reading and writing disabilities can still hone their skills in a new language.

## How we built it

We built the frontend UI using React, Javascript, HTML and CSS.

For the Backend, we used Node.js and Express.js. We made use of Google Cloud's speech-to-text API. We also utilized Cohere's API to generate text using their LLM.

Finally, for user authentication, we made use of Firebase.

## Challenges we faced + What we learned

When you first open our web app, our homepage consists of a lot of information on our app and our target audience. From there the user needs to log in to their account. User authentication is where we faced our first major challenge. Third-party integration took us significant time to test and debug.

Secondly, we struggled with the generation of prompts for the user to repeat and using AI to implement that.

## Accomplishments that we're proud of

This was the first time for many of our members to be integrating AI into an application that we are developing so that was a very rewarding experience especially since AI is the new big thing in the world of technology and it is here to stay.

We are also proud of the fact that we are developing an application for individuals with learning disabilities as we strongly believe that everyone has the right to education and their abilities should not discourage them from trying to learn new things.

## What's next for ReadRight

As of now, ReadRight has the basics of the English language for users to study and get prompts from but we hope to integrate more languages and expand into a more widely used application. Additionally, we hope to integrate more features such as voice-activated commands so that it is easier for the user to navigate the application itself. Also, for better voice recognition, we should
## Inspiration

COVID revolutionized the way we communicate and work by normalizing remoteness. **It also created a massive emotional drought -** we weren't made to empathize with each other through screens. As video conferencing turns into the new normal, marketers and managers continue to struggle with remote communication's lack of nuance and engagement, and those with sensory impairments likely have it even worse.

Given our team's experience with AI/ML, we wanted to leverage data to bridge this gap. Beginning with the idea to use computer vision to help sensory impaired users detect emotion, we generalized our use-case to emotional analytics and real-time emotion identification for video conferencing in general.

## What it does

In MVP form, empath.ly is a video conferencing web application with integrated emotional analytics and real-time emotion identification. During a video call, we analyze the emotions of each user sixty times a second through their camera feed.

After the call, a dashboard is available which displays emotional data and data-derived insights such as the most common emotions throughout the call, changes in emotions over time, and notable contrasts or sudden changes in emotion

**Update as of 7.15am: We've also implemented an accessibility feature which colors the screen based on the emotions detected, to aid people with learning disabilities/emotion recognition difficulties.**

**Update as of 7.36am: We've implemented another accessibility feature which uses text-to-speech to report emotions to users with visual impairments.**

## How we built it

Our backend is powered by Tensorflow, Keras and OpenCV. We use an ML model to detect the emotions of each user, sixty times a second. Each detection is stored in an array, and these are collectively stored in an array of arrays to be displayed later on the analytics dashboard. On the frontend, we built the video conferencing app using React and Agora SDK, and imported the ML model using Tensorflow.JS.

## Challenges we ran into

Initially, we attempted to train our own facial recognition model on 10-13k datapoints from Kaggle - the maximum load our laptops could handle. However, the results weren't too accurate, and we ran into issues integrating this with the frontend later on. The model's still available on our repository, and with access to a PC, we're confident we would have been able to use it.

## Accomplishments that we're proud of

We overcame our ML roadblock and managed to produce a fully-functioning, well-designed web app with a solid business case for B2B data analytics and B2C accessibility.

And our two last minute accessibility add-ons!

## What we learned

It's okay - and, in fact, in the spirit of hacking - to innovate and leverage on pre-existing builds. After our initial ML model failed, we found and utilized a pretrained model which proved to be more accurate and effective.

Inspiring users' trust and choosing a target market receptive to AI-based analytics is also important - that's why our go-to-market will focus on tech companies that rely on remote work and are staffed by younger employees.

## What's next for empath.ly

From short-term to long-term stretch goals:

* We want to add on AssemblyAI's NLP model to deliver better insights. For example, the dashboard could highlight times when a certain sentiment in speech triggered a visual emotional reaction from the audience.
* We want to port this to mobile and enable camera feed functionality, so those with visual impairments or other difficulties recognizing emotions can rely on our live detection for in-person interactions.
* We want to apply our project to the metaverse by allowing users' avatars to emote based on emotions detected from the user.
## Inspiration

We've all had that experience with our parents and grandparents asking us for help with the computer or smartphone, especially being at home in the pandemic with most meetings being virtual. This brought to mind the technological barriers between us and the older generation, as well as more so with lower socioeconomic groups who may not have access to the same privileges we have. Thus we got to thinking how can we increase accessibility of technology, and an idea is that people would need immediate help and would prefer an interface where they can talk to someone, but also more comprehensive educational materials are available to learn the technologies. Since what older people generally ask is 'can you help me with [insert technological issue]', the idea of CanYouHelpWith.Tech website was born.

## What it does

Our website has 4 main functions:

1. offer immediate tech support through search options to narrow your problem and video call and chat with our volunteers 24/7,
2. provide free live and recorded educational workshops to the general public,
3. become a volunteer and
4. make a donation.

## How we built it

Prototyping:
Figma

Website:

* React web app
* Node.js
* JavaScript, CSS, HTML
* Google Calendar API
* Goggle Auth2 User login
* Google Speech to text API that utilizes ML
* Twilio API
* Heroku Deployment
* Domain.com for a custom domain

## Challenges we ran into

The biggest challenge was that most of our teammates haven’t used the technologies such as React, Twilio and Google Authorization before, so we had to learn these as we used them. As expected, some errors also came up when running the code, which we discussed and worked through together to solve. Another issue that came up was with typescript files interfering with the javascript files, but this was resolved by removing the typescript.

## Accomplishments that we're proud of

* Built a mostly functional website!
* Learned React and Twilio
* Utilized features of Google Calendar, Google Speech to text and Twilio video call and chat

## What we learned

* How to build a website
* New Technologies: React, Twilio, Google technologies API
* How to merge conflicts with Git

## What's next for CanYouHelpWithMy.Tech

* Text-to-speech to aid those with vision issues
* Continue fixing bugs
* Optimize/update UI
* Recruit volunteers
* Advertising