## Inspiration

The challenge of delivering speeches with clarity and confidence inspired us to create SpeechSage. Furthermore, improved communication skills lead to better job opportunities, which is important for college undergrads. Whether for presentations, interviews, or public speaking, we wanted to offer a tool to help users perfect their communication skills.

## What it does

SpeechSage analyzes speech input and provides real-time feedback on content, tone, and pace. It helps users refine their speeches, ensuring they deliver confidently and clearly.

## How we built it

We built SpeechSage using React for the front end and integrated AI models from Google Gemini for content analysis. Additionally, HumeAI was used to analyze the emotions conveyed in the speech. The app provides a user-friendly interface for easy input and real-time feedback. A crucial part of the development involved managing the integration of these APIs, ensuring seamless communication between the speech input, emotion detection, content analysis, and the feedback delivery system.

## Challenges we ran into

Integrating the APIs and ensuring accurate real-time feedback was a significant hurdle. One challenge was how the APIs accepted parameters, as well as the security layer in certain APIs, which restricted deployment on client websites. Managing the flow between user input, processing time, and feedback delivery posed additional technical challenges. However, we are proud that we managed to obtain meaningful results despite these constraints.

## Accomplishments that we're proud of

We’re proud to have built a functional prototype that effectively provides helpful feedback on speech. Successfully integrating both HumeAI for emotional analysis and Google Gemini for content analysis was a major technical achievement. The emotional feedback helps users align their tone with their content, and this dual analysis ensures a holistic improvement in public speaking.

## What we learned

We learned the importance of user experience in AI-driven applications, particularly in speech analysis. Ensuring that both emotional and content feedback is timely, clear, and helpful taught us how to handle complex interactions between AI and front-end systems. Additionally, the security challenges around API deployment reinforced our understanding of how to handle API security in web applications.

## What's next for SpeechSage

We plan to make SpeechSage even easier to use with improved user experience through constant feedback and optimization. Future updates will include more detailed feedback, expanding language support, and refining both HumeAI and Google Gemini models to better adapt to various speaking styles and emotional contexts. We also aim to simplify API integration to make the system more robust and secure for client-side deployments.
## Inspiration

Since we are a group of students, we have been seeing our peers losing motivation for school due to un-engaging lectures. Young adults are more stressed due to added responsibilities of taking care of younger siblings at home and supporting their family financially. Thus, students who have more responsibilities due to the pandemic miss a lot of classes, and they clearly don't have a lot of time to re-watch a one-hour lecture that they've missed.

This was me, during the earlier months of the pandemic. By having to work extra hours due to the financial impacts of the Coronavirus, alongside the inaccessibility of the internet when being outside, I missed the most important classes to finish my requirements for my degree.

That's where the inspiration of this project came from. I personally know people in different fields facing the same issue, and with my team, I wanted to help them out.

## What it does

By taking an audio file as the input, we use the Google Cloud API's function to turn the audio into text. We then analyze that text to determine the main topics by word frequency which we display to the user. We then display all sentences containing words with the highest frequency to the user.

## How we built it

First, we laid out the wireframe on Figma; after further discussion, the dev team went on working on the backend while the design team worked on the high-fidelity prototype.

After the high-fidelity prototype was handed off to the developers, the dev team then built the frontend aspect of our product to allow the user to select audios which they want to condense.

## Challenges we ran into

While building the backend, we ran into numerous bugs when developing algorithms to detect the main topics of the audio converted text. After resolving that issue, we had to figure out how to use the Google Cloud API to convert the audio files into text to pass into our processing algorithms. Finally, we had to find a way to connect our website to our backend containing our text-processing algorithms.

## Accomplishments that we're proud of

Our team figured out to convert speech to text and to display the output of our text processing algorithms to the user. Our team is also proud of creating a website that displays what our product does, acting as a portfolio of our product.

## What we learned

We learned how to utilize APIs, develop algorithms to process our text using patterns, debugging our code while learning under a time limit with new teammates.

## What's next for Xalta

* Integration with Zoom and Canvas for a more seamless user experience
* A desktop/mobile native app for stability
## Inspiration

Patients usually have to go through multiple diagnosis before getting the right doctor. With the astonishing computational power we have today, we could use predictive analysis to suggest the patients' potential illness.

## What it does

Clow takes a picture of the patient's face during registration and run it through an emotion analysis algorithm. With the "scores" that suggest the magnitude of a certain emotional trait, Clow matches these data with the final diagnosis result given by the doctor to predict illnesses.

## How we built it

We integrated machine learning and emotion analysis algorithms from Microsoft Azure cloud services on our Ionic-based app to predict the trends. We "trained" our machine by pairing the "scores" of images of sick patients with its illness, allowing it to predict illnesses based on the "scores".

## Challenges we ran into

All of us are new to machine learning and this has proved to be a challenge to all of us. Fortunately, Microsoft's representative was really helpful and guided us through the process. We also had a hard time writing the code to upload the image taken from the camera to a cloud server in order to run it through Microsoft's emotion analysis API, since we have to encode the image before uploading it.

## Accomplishments that we're proud of

Learning a new skill over a weekend and deploy it on a working prototype ain't easy. We did that, not one but two skills, over a weekend. And it's machine learning and emotion analysis. And they are actually the main components that powers our product.

## What we learned

We all came in with zero knowledge of machine learning and now we are able to walk away with a good idea of what it is. Well, at least we can visualize it now, and we are excited to work with machine learning and unleash its potential in the future.

## What's next for Clow

Clow needs the support of medical clinics and hospitals in order to be deployed. As the correlation between emotion and illness is still relatively unproven, research studies have to be done in order to prove its effectiveness. It may not be effectively produce results in the beginning, but if Clow analyzes thousands of patients' emotion and illness, it can actually very accurately yield these results.