# JARVIS-MUSIC

Have a song stuck in your head but can't sing it? Use our web app to play the music that's on your mind!

Overall, we are building a web application for users to find songs by lirycs or any information that is related to your song.

In our web home page, you press the audio recording button to sing your songs, or tell something about your song.
We use google speech to text API to convert your voice to text and show it to you.
Of course, you can always input the text directly, if you are uncomfortable about saying anything.
The text is transformed to our server side. We have a machine learning algorithm to search for the most related song.

Here is our algorithm to search the song. We firstly download tons of songs using the Genius API. We build a word filtering API using the standard library (stdlib). The word filtering API filters bad words (such as "fuck") in the lyrics of the songs. We then index the songs to our database. Given your input text, i.e., a query, our unsupervised algorithm aims to find the most relevant song. We use the Vector Space Model and doc2vec to build our unsupervised algorithm.
Basically, Vector Space Model locates the exact words that you want to search and doc2vec extracts the semantic meaning of your input text and the lyrics, such that we are not lossing any information. We give a score of each song, denoting how relevant the song is to the input text. We rank the songs based the returned scores. and return the top five songs. We not only return song name and singer name, but also return YouTube links for you to check out!
## Inspiration

We wanted to create a tool that would improve the experience of all users of the internet, not just technically proficient ones. Once fully realized, our system could allow groups such as blind people and the elderly a great deal of utility, as it allows for navigation by simple voice command with only minor setup from someone else. We could also implement function sharing to allow users to share their ideas and tools with everyone else or allow companies to define defaults on their own webpages to improve the user experience.

Created with the intention to make browsing the web easier, if not possible for the visually impaired. Access to the internet may soon be considered a fundamental human right- it should be made accessible to everyone regardless of their abilities.

## What it does

A chrome extension that allows a user to speak a phrase(e.g. "Search for guitar songs" on Youtube), then enter a series of actions. Our extension will store the instructions entered, then use a combination of STT and natural language processing to generalize the query for the site and all permutations of the same structure. We use Microsoft LUIS, which at a baseline allows for synonyms. The more it is used, the better this becomes, so it could expand to solve "find piano music" as well. We are also in the process of developing a simple interface to allow users to easily define their custom instruction sets.

## How we built it

We used webkit Speech to Text in a chrome plugin to create sentences from recordings. We also created a system to track mouse and keyboard inputs in order to replicate them. This info was passed to a StdLib staging area that processes data and manages Microsoft LUIS to interpret the inputted sentence. This is then passed back to the plugin so it knows which sequence of actions to perform. Our project has a system to generalize "entities" i.e. the variables in an instruction (i.e. "guitar songs").

## Challenges we ran into

* No native APIs for several UX/UI elements. Forced to create workarounds and hack bits of code together.
* Making the project functions easy for users to follow and understand.

## Accomplishments that we're proud of

Our team learned to use an unfamiliar system with an entirely different paradigm from traditional web hosting, and how to manage its advantages and disadvantages on the fly while integrating with several other complex systems

## What we learned

It is a better strategy to iterate on outwards from the simplest core of your system rather than aim big. We had to cut features, which meant we sunk unnecessary time into development initially. We also learned all about FAAS and serverless hosting, and about natural language processing.

## What's next for Quack - Voice Controlled Action Automation for Chrome
## Inspiration

We had a wine and cheese last week.

## Challenges I ran into

A W S + python 3 connecting to domain

## What's next for Whine and Cheese

A team wine and cheese