## Inspiration

The idea for Iris was born out of a mutual agreement on the need for interconnectivity between different languages. While technology has continued to allow more and more connectivity across far distances, one of the most significant barriers to this connectivity is the language barrier. We resolve this issue with Iris.

## What it does

Iris is a full-stack, web-based that creates translated transcriptions of live audio. A class instructor, keynote speaker, or any other user can make a room, have other users join the room, and immediately send live translated transcriptions to all of the users in the room.

## How we built it

The frontend of Iris was constructed using React, React-Router, and Javascript. The backend was made in Python and interacted with AssemblyAI API and the Google Translate API. The frontend and backend communicate with each other using WebSockets.

## Challenges we ran into

Due to the frontend and backend being written in different languages, converting the recorded audio to an acceptable format for the AssemblyAI API was challenging and took many hours of testing and debugging. While we made ample use of the documentation provided by AssemblyAI, there were no snippets that showcased inter-language interactions. Trial and error eventually prevailed, but not without a challenge.

Setting up Google Translate API was also a challenge. There was a tutorial on setting up Google Translate API, but we executed the code via the Google Cloud console. The commands given in the tutorial only worked with the Google Cloud SDK, and there wasn't a guide on converting those commands into Python. This was frustrating as we needed to debug the translation code outside of the Google Cloud console.

## Accomplishments that we're proud of

Iris accomplishes what we set out to do. It supports many languages and even allows for multiple simultaneous rooms displaying translated text. Overall, we are very proud of what we have accomplished with Iris especially considering the 36 hour timeframe.

## What we learned

We learned how to employ WebSockets for a continuous connection between the front and backend throughout this hackathon. Additionally, everyone also learned how to use the AssemblyAI and Google Cloud APIs and their documentation to help us build Iris.

## What's next for Iris

One potential avenue that we could take is implementing live video feeds in the future. Additionally, allowing hosts to upload recordings of their files for translation through an upload menu can further increase accessibility for Iris users.
## Inspiration 🌎

One of the challenges in our multicultural country is providing healthcare to non-verbal or non-English speaking patients.

Health care professionals often face a challenge when communicating with these patients. Bedside workers are not provided a translator and often rely on the patient's family for communication. This leads to miscommunication with the patient and misinformation from the family that may not be entirely truthful when answering questions involving medical history.

This also creates a problem when the nurses are trying to fully inform their patients before receiving their consent for a treatment. Nurses are liable is they don't received informed consent and communicating with non-verbal or non-English speaking patients can become increasing stressful for this reason.

Our team wanted address all of these problems while improving the efficiency and quality of life for these hard working, under staffed professionals.

## What does it do 🤔

Our application provides a system to help healthcare workers communicate with non-verbal and non-English speaking patients in a user friendly manner using real-time transcription and translating, graphical pain indicators, and visual symptom assessment surveys.

## How we built it 🔨

Using AssemblyAI + Google Translate API we created a real-time transcription and translating system that can cross the language barrier between the patient and the healthcare worker.

Interactive healthcare diagrams made using HTML and JavaScript designed to be simple and user-friendly were constructed to help patients visually communicate with professionals quickly.

All of the data is stored for later use in a secure hospital database to help keep track of patients' progress and make for easy data sharing between shifts of nurses.

## Challenges we ran into 🔎

Configuring the translations API and incorporating it into a browser environment proved to be very difficult when developing the backend. After hours of reading documentation, consulting mentors, and trying different approaches we finally got these tools to work seamlessly for our subtitle generator.

## Accomplishments that we're proud of 💪

We were able to implement real-time captioning using AssemblyAI and translation using the Google Translate API. We are also proud that we made a fully functioning web application in only 36 hours without the use of a framework. We think our program can provide real benefits to the healthcare industry.

## What we learned 🧠

We all learned how to use AssemblyAI and some of us learned JavaScript for the first time. We got to build on our UI development skills and refine our knowledge of databases.

## Looking Forward ⏩

We plan to implement foreign to English language translation to improve communication between the patient and nurse. With more time, we would have added functionality for nurses to customize the symptoms questionnaire and patient needs menu to further improve user experience.
## Inspiration

Learning new languages is an amazing way to keep one mentally sharp, meet and connect with new people, and even delaying Alzheimer's. But learning a new one can be a daunting and seemingly unattainable challenge. To this end, we created **LanguageShare**, a platform where individuals can hold themselves accountable by teaching each other new languages, and utilizes the power of AI to create an accessible and user-first interface that prioritizes connection and learning.

## What it does

**LanguageShare** is a **React** web application which is used for individuals to learn a language of their choice and teach someone their own. We assign users an ELO score and match individuals on criteria of score and language(s) of choice that they wish to learn. Once users are connected, they are placed in a video call with AI-driven live transcription and translation, so that the two users can teach each other their language and have captioning on the side to help them learn and understand new phrases and words.

## How we built it

We built the application with **React**, **JavaScript**, and **TypeScript**. For video conferencing, we used **VideoSDK**, a web RTC-based video conferencing library, and we combined Google Cloud's Speech-to-Text and Translation API's for our live, AI-driven transcription and translation functionality. On the backend, we used **Express.js** and **Socket.io** to connect and store users, and designed and implemented mathematical and greedy algorithms to assign users accurate ELO ratings and match them appropriately.

## Challenges we ran into

Our biggest challenge was sending the audio to the Google Cloud API's in an appropriate format to then be transcribed and translated. Getting the API output to then appear in a readable and concise manner on our UI was also a big challenge.

## Accomplishments that we're proud of

We are proud of getting all our different components working together and to have accurate and real-time translation working effectively in the frontend.

## What we learned

We all learned much more about web development, React, and building video conferencing applications, which none of us had any prior experience with.

## What's next for LanguageShare

* Adding support for more languages
* Streamlining the UI and user-profile creation process
* Allow users to rate each others fluency after calls, leading to more nuanced ELO rating calculations
* Live Chat features during calls