## Inspiration

In today's world, Public Speaking is one of the greatest skills any individual can have. From pitching at a hackathon to simply conversing with friends, being able to speak clearly, be passionate and modulate your voice are key features of any great speech. To tackle this problem of becoming a better public speaker, we created Talky.

## What it does

It helps you improve your speaking skills by giving you suggestions based on what you said to the phone. Once you finish presenting your speech to the app, an audio file of the speech will be sent to a flask server running on Heroku. The server will analyze the audio file by examining pauses, loudness, accuracy and how fast user spoke. In addition, the server will do a comparative analysis with the past data stored in Firebase. Then the server will return the performance of the speech.The app also provides the community functionality which allows the user to check out other people’s audio files and view community speeches.

## How we built it

We used Firebase to store the users’ speech data. Having past data will allow the server to do a comparative analysis and inform the users if they have improved or not.
The Flask server uses similar Audio python libraries to extract meaning patterns: Speech Recognition library to extract the words, Pydub to detect silences and Soundfile to find the length of the audio file.

On the iOS side, we used Alamofire to make the Http request to our server to send data and retrieve a response.

## Challenges we ran into

Everyone on our team was unfamiliar with the properties of audio, so discovering the nuances of wavelengths in particular and the information it provides was challenging and integral part of our project.

## Accomplishments that we're proud of

We successfully recognize the speeches and extract parameters from the sound file to perform the analysis.
We successfully provide the users with an interactive bot-like UI.
We successfully bridge the IOS to the Flask server and perform efficient connections.

## What we learned

We learned how to upload audio file properly and process them using python libraries.
We learned to utilize Azure voice recognition to perform operations from speech to text.
We learned the fluent UI design using dynamic table views.
We learned how to analyze the audio files from different perspectives and given an overall judgment to the performance of the speech.

## What's next for Talky

We added the community functionality while it is still basic. In the future, we can expand this functionality and add more social aspects to the existing app.
Also, the current version is focused on only the audio file. In the future, we can add the video files to enrich the post libraries and support video analyze which will be promising.
## Inspiration

Many people feel unconfident, shy, and/or awkward doing interview speaking. It can be challenging for them to know how to improve and what aspects are key to better performance. With Talkology, they will be able to practice in a rather private setting while receiving relatively objective speaking feedback based on numerical analysis instead of individual opinions. We hope this helps more students and general job seekers become more confident and comfortable, crack their behavioral interviews, and land that dream offer!

## What it does

* Gives users interview questions (behavioural, future expansion to questions specific to the job/industry)
* Performs quantitative analysis of users’ responses using speech-to-text & linguistic software package praat to study acoustic features of their speech
* Displays performance metrics with suggestions in a user-friendly, interactive dashboard

## How we built it

* React/JavaScript for the frontend dashboard and Flask/Python for backend server and requests
* My-voice-analysis package for voice analysis in Python
* AssemblyAI APIs for speech-to-text and sentiment analysis
* MediaStream Recording API to get user’s voice recordings
* Figma for the interactive display and prototyping

## Challenges we ran into

We went through many conversations to reach this idea and as a result, only started hacking around 8AM on Saturday. On top of this time constraint layer, we also lacked experience in frontend and full stack development. Many of us had to spend a lot of our time debugging with package setup, server errors, and for some of us even M1-chip specific problems.

## Accomplishments that we're proud of

This was Aidan’s first full-stack application ever. Though we started developing kind of late in the event, we were able to pull most of the pieces together within a day of time on Saturday. We really believe that this product (and/or future versions of it) will help other people with not only their job search process but also daily communication as well. The friendships we made along the way is also definitely something we cherish and feel grateful about <3

## What we learned

* Aidan: Basics of React and Flask
* Spark: Introduction to Git and full-stack development with sprinkles of life advice
* Cathleen: Deeper dive into Flask and React and structural induction
* Helen: Better understanding of API calls & language models and managing many different parts of a product at once

## What's next for Talkology

We hope to integrate computer vision approaches by collecting video recordings (rather than just audio) to perform analysis on hand gestures, overall posture, and body language. We also want to extend our language analysis to explore novel models aimed at performing tone analysis on live speech. Apart from our analysis methods, we hope to improve our question bank to be more than just behavioural questions and better cater to each user's specific job demands. Lastly, there are general loose ends that could be easily tied up to make the project more cohesive, such as integrating the live voice recording functionality and optimizing some remaining components of the interactive dashboard.
## Inspiration

We wanted to solve a problem that was real for us, something that we could get value out of. We decided upon Vocally as it solves an issue faced by a lot of people during job interviews, presentations, and other occasions which include speaking in a clear and concise manner. The problem was that it takes a long time to record yourself and re-listen to it just to spot any sentence fillers like "um" or "like". We would like to make it easier to display statistics of one's speech.

## What it does

The user clicks the record button and starts speaking. The application first converts speech-to-text using React's built-in speech recognition. After analyzing the results various text processing techniques (e.g. sentiment analysis), it displays feedback.

## How we built it

* First, we needed to see how keywords could be extracted from an audio recording in the back-end. We settled with React's speech-to-text feature.
* Next, we created API endpoints in Flask (a python web framework) for the React app to make requests from.
* Fuzzy string matching, grammatical, and sentiment analysis were used to process and return the stats to the user using data visualization.
* The last task was deployment to the pythonanywhere.com domain for demo testing purposes.

## Challenges I ran into

Using flask as an API was easy, but we initially tried to host it on GCP, which proved to be difficult as our firewall rules were not configured properly. We moved onto pythonanywhere.com for hosting. For the front-end, we first decided to take a look at the Flutter framework to be able to make the application mobile accessible but the framework was introduced in 2018, and there were a lot of configuration issues that needed to be resolved.

## Accomplishments that we are proud of

Getting the sound recorder to work on the front-end took longer than expected, but the end result was very satisfying. We're proud that we actually achieved creating an end-to-end solution.

## What I learned

Exploring different framework options like Flutter, in the beginning, was a journey for us. The API that was created needed to delve deeper into the python programming language. We learned about various syntactical and natural language processing techniques.

## What's next for Vocally

We may re-explore the concept of natural language processing, perhaps build our own algorithm from scratch and do more over a longer time period.