## Inspiration

We've all been there - racing to find an empty conference room as your meeting is about to start, struggling to hear a teammate who's decided to work from a bustling coffee shop, or continuously muting your mic because of background noise.

As the four of us all conclude our internships this summer, we’ve all experienced these over and over. But what if there is a way for you to simply take meetings in the middle of the office…

## What it does

We like to introduce you to Unmute, your solution to clear and efficient virtual communication. Unmute transforms garbled audio into audible speech by analyzing your lip movements, all while providing real-time captions as a video overlay. This means your colleagues and friends can hear you loud and clear, even when you’re not. Say goodbye to the all too familiar "wait, I think you're muted".

## How we built it

Our team built this application by first designing it on Figma. We built a React-based frontend using TypeScript and Vite for optimal performance. The frontend captures video input from the user's webcam using the MediaRecorder API and sends it to our Flask backend as a WebM file. On the server side, we utilized FFmpeg for video processing, converting the WebM to MP4 for wider compatibility. We then employed Symphonic's API to transcribe visual cues.

## Challenges we ran into

Narrowing an idea was one of the biggest challenges. We had many ideas, including a lip-reading language course, but none of them had a solid use case. It was only after we started thinking about problems we encountered in our daily lives did we find our favorite project idea.

Additionally, there were many challenges on the technical side with using Flask and uploading and processing videos.

## Accomplishments that we're proud of

We are proud that we were able to make this project come to life.

## Next steps

Symphonic currently does not offer websocket functionality, so our vision of making this a real-time virtual meeting extension is not yet realizable. However, when this is possible, we are excited for the improvements this project will bring to meetings of all kinds.
## Inspiration

Our project was inspired by people who don't learn the "old school" way. Some people learn best by reciting things out loud, but the problem with that is sometimes it's too quiet to talk, and sometimes it's too loud to talk.

## What it does

Our project is a website that allows you to create a group with your classmates/coworkers, in which you can record video of yourself silently mouthing out whatever lecture or book you're reading, allowing you to pay full attention without worrying about writing notes, a loud environment, or an environment in which you're not allowed to talk. Our project reads your lips and puts it into text, creating notes for the video you take. Then, with your notes, we summarize your key points and build a quiz so you can focus on the important parts of your work.

## How we built it

For the front end of our website, we used React JS and Chakra. We used Auth0 to create an account system for our project. We used Symphonic Labs' API to read your lips and convert your silent words to text. We used MongoDB to save all your notes to a database in your group. We then used a Cohere API to summarize the key points of the text and make a quiz out of it. We also used an Open AI API to convert mp3 files to text, in case you wanted to record a lecturer or friend speaking, and then convert it to notes and quizzes.

## Challenges we ran into

We tried to get live video transcribing with Symphonic Lab's API using Open CV, but since Open CV was giving us frames of the video, Symphonic Lab's API wasn't able to transcribe it. Instead, we switched to allowing users to record their videos on the site, immediately download them and upload it for transcribing.

## Accomplishments that we're proud of

We are very proud of combining all the complicated API's to create a complete and flowing product, something that all of us will definitely be using for school.

## What we learned

This was our group's first time using more than one API for a project, but it ended up with a website that performs a useful and complicated function, and we will be using multiple APIs in the future!

## What's next for Studying with Hack the North
## Inspiration

There are millions of people around the world who have a physical or learning disability which makes creating visual presentations extremely difficult. They may be visually impaired, suffer from ADHD or have disabilities like Parkinsons. For these people, being unable to create presentations isn’t just a hassle. It’s a barrier to learning, a reason for feeling left out, or a career disadvantage in the workplace. That’s why we created **Pitch.ai.**

## What it does

Pitch.ai is a web app which creates visual presentations for you as you present. Once you open the web app, just start talking! Pitch.ai will listen to what you say and in real-time and generate a slide deck based on the content of your speech, just as if you had a slideshow prepared in advance.

## How we built it

We used a **React** client combined with a **Flask** server to make our API calls. To continuously listen for audio to convert to text, we used a react library called “react-speech-recognition”. Then, we designed an algorithm to detect pauses in the speech in order to separate sentences, which would be sent to the Flask server.

The Flask server would then use multithreading in order to make several API calls simultaneously. Firstly, the **Monkeylearn** API is used to find the most relevant keyword in the sentence. Then, the keyword is sent to **SerpAPI** in order to find an image to add to the presentation. At the same time, an API call is sent to OpenAPI’s GPT-3 in order to generate a caption to put on the slide. The caption, keyword and image of a single slide deck are all combined into an object to be sent back to the client.

## Challenges we ran into

* Learning how to make dynamic websites
* Optimizing audio processing time
* Increasing efficiency of server

## Accomplishments that we're proud of

* Made an aesthetic user interface
* Distributing work efficiently
* Good organization and integration of many APIs

## What we learned

* Multithreading
* How to use continuous audio input
* How to use React hooks, Animations, Figma

## What's next for Pitch.ai

* Faster and more accurate picture, keyword and caption generation
* "Presentation mode”
* Integrate a database to save your generated presentation
* Customizable templates for slide structure, color, etc.
* Build our own web scraping API to find images