# talko

Hello! talko is a project for nwHacks 2022.

Interviews can be scary, but they don't have to be! We believe that practice and experience is what gives you the
confidence you need to show interviewers what you're made of. talko is made for students and new graduates who want to
learn to fully express their skills in interviews. With everything online, it's even more important now that you can
get your thoughts across clearly virtually.

As students who have been and will be looking for co-ops, we know very well how stressful interview season can be;
we took this as our source of inspiration for talko.

talko is an app that helps you practice for interviews. Record and time your answers to interview questions to get
feedback on how fast you're talking and view previous answers.

## Features

* View answer history for previous answers- playback recordings, words per minute, standard deviation of talking speed and overall answer quality.
* Integrated question bank with a variety of topics.
* Skip answers you aren't ready to answer.
* Adorable robots!!

## Technical Overview

For Talko’s front-end, we used React to create a web app that can be used on both desktop and mobile devices.

We used Figma for the wireframing and Adobe Fresco for some of the aesthetic touches and character designs.

We created the backend using Nodejs and Express. The api handles uploading, saving and retrieving recordings, as well as
fetching random questions from our question bank. We used Google Cloud Firestore to save data about past answers, and Microsoft
Azure to store audio files and use speech-to-text on our clips. In our api, we calculate the average words per minute over
the entire answer, as well as the variance in the rate of speech.

## Challenges & Accomplishments

Creating this project in just 24 hours was quite the challenge! While we have worked with some of these tools before,
it was our first time working with Microsoft Azure. We're really proud of what we managed to put together over this weekend.

Another issue we had is that it can take a while to get speech-to-text results from Azure. We wanted to send
a response back to the frontend quickly, so we decided to calculate the rate of speech variance afterwards and
patch our data in Firestore.

## What's next for talko?

* Tagged questions: get questions most relevant to your industry
* Better answer analysis: use different NLP APIs and assess the text to give better stats and pointers
	+ Are there lots of pauses and filler words in the answer?
	+ Is the answer related to the question?
	+ Given a job description selected or supplied by the user, does the answer cover the keywords?
	+ Is the tone of the answer formal, assertive?
* View answer history in more detail: option to show transcript and play back audio recordings
* Settings to personalize your practice experience: customize number of questions and answer time limit.

## Built using

![image](https://img.shields.io/badge/Node.js-339933?style=for-the-badge&logo=nodedotjs&logoColor=white)
![image](https://img.shields.io/badge/React-20232A?style=for-the-badge&logo=react&logoColor=61DAFB)
![image](https://img.shields.io/badge/Express.js-000000?style=for-the-badge&logo=express&logoColor=white)
![image](https://img.shields.io/badge/microsoft%20azure-0089D6?style=for-the-badge&logo=microsoft-azure&logoColor=white)
![image](https://img.shields.io/badge/firebase-ffca28?style=for-the-badge&logo=firebase&logoColor=black)

![drawing](https://github.com/nwhacks-2022/.github/blob/main/assets/rainbow.png?raw=true)

### Thanks for visiting!
## Inspiration

Being a student of the University of Waterloo, every other semester I have to attend interviews for Co-op positions. Although it gets easier to talk to people, the more often you do it, I still feel slightly nervous during such face-to-face interactions. During this nervousness, the fluency of my conversion isn't always the best. I tend to use unnecessary filler words ("um, umm" etc.) and repeat the same adjectives over and over again. In order to improve my speech through practice against a program, I decided to create this application.

## What it does

InterPrep uses the IBM Watson "Speech-To-Text" API to convert spoken word into text. After doing this, it analyzes the words that are used by the user and highlights certain words that can be avoided, and maybe even improved to create a stronger presentation of ideas. By practicing speaking with InterPrep, one can keep track of their mistakes and improve themselves in time for "speaking events" such as interviews, speeches and/or presentations.

## How I built it

In order to build InterPrep, I used the Stdlib platform to host the site and create the backend service. The IBM Watson API was used to convert spoken word into text. The mediaRecorder API was used to receive and parse spoken text into an audio file which later gets transcribed by the Watson API.

The languages and tools used to build InterPrep are HTML5, CSS3, JavaScript and Node.JS.

## Challenges I ran into

"Speech-To-Text" API's, like the one offered by IBM tend to remove words of profanity, and words that don't exist in the English language. Therefore the word "um" wasn't sensed by the API at first. However, for my application, I needed to sense frequently used filler words such as "um", so that the user can be notified and can improve their overall speech delivery. Therefore, in order to implement this word, I had to create a custom language library within the Watson API platform and then connect it via Node.js on top of the Stdlib platform. This proved to be a very challenging task as I faced many errors and had to seek help from mentors before I could figure it out. However, once fixed, the project went by smoothly.

## Accomplishments that I'm proud of

I am very proud of the entire application itself. Before coming to Qhacks, I only knew how to do Front-End Web Development. I didn't have any knowledge of back-end development or with using API's. Therefore, by creating an application that contains all of the things stated above, I am really proud of the project as a whole. In terms of smaller individual accomplishments, I am very proud of creating my own custom language library and also for using multiple API's in one application successfully.

## What I learned

I learned a lot of things during this hackathon. I learned back-end programming, how to use API's and also how to develop a coherent web application from scratch.

## What's next for InterPrep

I would like to add more features for InterPrep as well as improve the UI/UX in the coming weeks after returning back home. There is a lot that can be done with additional technologies such as Machine Learning and Artificial Intelligence that I wish to further incorporate into my project!
## Inspiration

In today's world, Public Speaking is one of the greatest skills any individual can have. From pitching at a hackathon to simply conversing with friends, being able to speak clearly, be passionate and modulate your voice are key features of any great speech. To tackle this problem of becoming a better public speaker, we created Talky.

## What it does

It helps you improve your speaking skills by giving you suggestions based on what you said to the phone. Once you finish presenting your speech to the app, an audio file of the speech will be sent to a flask server running on Heroku. The server will analyze the audio file by examining pauses, loudness, accuracy and how fast user spoke. In addition, the server will do a comparative analysis with the past data stored in Firebase. Then the server will return the performance of the speech.The app also provides the community functionality which allows the user to check out other people’s audio files and view community speeches.

## How we built it

We used Firebase to store the users’ speech data. Having past data will allow the server to do a comparative analysis and inform the users if they have improved or not.
The Flask server uses similar Audio python libraries to extract meaning patterns: Speech Recognition library to extract the words, Pydub to detect silences and Soundfile to find the length of the audio file.

On the iOS side, we used Alamofire to make the Http request to our server to send data and retrieve a response.

## Challenges we ran into

Everyone on our team was unfamiliar with the properties of audio, so discovering the nuances of wavelengths in particular and the information it provides was challenging and integral part of our project.

## Accomplishments that we're proud of

We successfully recognize the speeches and extract parameters from the sound file to perform the analysis.
We successfully provide the users with an interactive bot-like UI.
We successfully bridge the IOS to the Flask server and perform efficient connections.

## What we learned

We learned how to upload audio file properly and process them using python libraries.
We learned to utilize Azure voice recognition to perform operations from speech to text.
We learned the fluent UI design using dynamic table views.
We learned how to analyze the audio files from different perspectives and given an overall judgment to the performance of the speech.

## What's next for Talky

We added the community functionality while it is still basic. In the future, we can expand this functionality and add more social aspects to the existing app.
Also, the current version is focused on only the audio file. In the future, we can add the video files to enrich the post libraries and support video analyze which will be promising.