## Inspiration

The music industry has gone stagnant. Artists at the top stay at the top, and many artists stay out of sight. Looking for new music is difficult because it is either an iteration of the music everyone has already heard or it is lost in the sea of the internet. Not only do the listeners suffer from this– so do the artists. Many work very hard but never get the exposure they deserve. Crescendo seeks to leverage the power of computation to revitalize the flow that existed in the music industry just decades ago.

## How it works

Crescendo uses the k-nearest neighbors algorithm along with collaborative filtering trained on data from SoundCloud to serve music to users by artists that are likely to be good but are still under the radar. As the user interacts with the website, saving certain songs and discarding others, the back-end "learns" the preferences of the user and offers them music liked by other users of similar taste. Collaborative filtering is a common algorithm used by recommender systems to determine the similarity of users with respect to products. Our implementation uses gradient decent to determine which artists our users might most enjoy. In order to escape the chicken-and-egg problem of needing user data to find quality music and needing quality music to draw data-generating users, we trained our collaborative-filtering matrices on scraped SoundCloud user-favorites data.

## Challenges, and what we learned

Turns out machine learning involves some very challenging math. We learned a lot about how machine learning actually works, and how to coordinate different techniques (such as k-nearest neighbors with collaborative filtering) as well as different frameworks (pandas, tensorflow, etc.) in order to build a multi-stage algorithm. As collaborative filtering is a large area of study, with various implementations, we had to read through a lot of papers before diving in to the algorithm we are currently using.
Working with big data from SoundCloud's API was also a challenge, as we often had to wait up to an hour to scrape a few hundred thousand favorites for use in the training step of our machine learning algorithm. We learned that writing algorithms to work on large data sets requires solid conceptual testing (asking yourself "will this work?"), since running the algorithms can take up significant time, a precious resource, and debugging can be a nightmare with thousands of lines of output.
## Inspiration

Everyone loves music, but finding new music can be a difficult task. We found that music websites, such as Spotify, emphasize recommendations similar to what we currently enjoy, but don't address the music that we don't enjoy. As a result, we wanted to come up with an app that recommends songs not necessarily always similar to what we like, but also distinct from our dislikes, introducing us to potentially new genres.

## What it does

Using a Tinder-like front-end, users can listen to ditties of popular songs from Spotify to discover new genres. By swiping left or right, users can influence subsequent song recommendations.

## How we built it

We created a recommendation algorithm using Scikit-learn and the Spotipy API that analyzes the user swipes to suggest new songs on our web app. Our web app was created using React.js and Flask, and our database of choice was CockroachDB.

## Challenges we ran into

This was the first full stack app we've made, so we ran into a lot of issues connecting the front-end and back-end components, as well as setting up our database. Additionally, it was a challenge to gather all of our data, as the API we used limited our request volume. In the end, with CockroachDB, we were able to connect and store data into it; however, we were unable to resolve querying from it on our "Liked Songs" page.

## Accomplishments that we're proud of

We are super proud of our front-end's interactability and our app's ability to dynamically generate cards that align with the user's interests!

## What we learned

We learned how to connect various python libraries (Scikit-learn, Flask, Spotipy, etc.) together to develop a robust full stack application.

## What's next for Dittycal

We hope to further develop the social aspect of this application, as given the time constraints of this hackathon, we focused on the core functionality (allowing users to find songs they like based previous ditties they've liked). It would be awesome to implement an account system, so that user song data can be analyzed and inputted to a model that implements collaborative filtering (think TikTok, Tinder, or even Netflix!).
## **Inspiration:**

Our inspiration stemmed from the realization that the pinnacle of innovation occurs at the intersection of deep curiosity and an expansive space to explore one's imagination. Recognizing the barriers faced by learners—particularly their inability to gain real-time, personalized, and contextualized support—we envisioned a solution that would empower anyone, anywhere to seamlessly pursue their inherent curiosity and desire to learn.

## **What it does:**

Our platform is a revolutionary step forward in the realm of AI-assisted learning. It integrates advanced AI technologies with intuitive human-computer interactions to enhance the context a generative AI model can work within. By analyzing screen content—be it text, graphics, or diagrams—and amalgamating it with the user's audio explanation, our platform grasps a nuanced understanding of the user's specific pain points. Imagine a learner pointing at a perplexing diagram while voicing out their doubts; our system swiftly responds by offering immediate clarifications, both verbally and with on-screen annotations.

## **How we built it**:

We architected a Flask-based backend, creating RESTful APIs to seamlessly interface with user input and machine learning models. Integration of Google's Speech-to-Text enabled the transcription of users' learning preferences, and the incorporation of the Mathpix API facilitated image content extraction. Harnessing the prowess of the GPT-4 model, we've been able to produce contextually rich textual and audio feedback based on captured screen content and stored user data. For frontend fluidity, audio responses were encoded into base64 format, ensuring efficient playback without unnecessary re-renders.

## **Challenges we ran into**:

Scaling the model to accommodate diverse learning scenarios, especially in the broad fields of maths and chemistry, was a notable challenge. Ensuring the accuracy of content extraction and effectively translating that into meaningful AI feedback required meticulous fine-tuning.

## **Accomplishments that we're proud of**:

Successfully building a digital platform that not only deciphers image and audio content but also produces high utility, real-time feedback stands out as a paramount achievement. This platform has the potential to revolutionize how learners interact with digital content, breaking down barriers of confusion in real-time. One of the aspects of our implementation that separates us from other approaches is that we allow the user to perform ICL (In Context Learning), a feature that not many large language models don't allow the user to do seamlessly.

## **What we learned**:

We learned the immense value of integrating multiple AI technologies for a holistic user experience. The project also reinforced the importance of continuous feedback loops in learning and the transformative potential of merging generative AI models with real-time user input.