## Inspiration

Learning a new instrument is hard. Inspired by games like Guitar Hero, we wanted to make a fun, interactive music experience but also have it translate to actually learning a new instrument. We chose the violin because most of our team members had never touched a violin prior to this hackathon. Learning the violin is also particularly difficult because there are no frets, such as those on a guitar, to help guide finger placement.

## What it does

Fretless is a modular attachment that can be placed onto any instrument. Users can upload any MIDI file through our GUI. The file is converted to music numbers and sent to the Arduino, which then lights up LEDs at locations corresponding to where the user needs to press down on the string.

## How we built it

Fretless is composed to software and hardware components.
We used a python MIDI library to convert MIDI files into music numbers readable by the Arduino. Then, we wrote an Arduino script to match the music numbers to the corresponding light. Because we were limited by the space on the violin board, we could not put four rows of LEDs (one for each string). Thus, we implemented logic to color code the lights to indicate which string to press.

## Challenges we ran into

One of the challenges we faced is that only one member on our team knew how to play the violin. Thus, the rest of the team was essentially learning how to play the violin and coding the functionalities and configuring the electronics of Fretless at the same time.
Another challenge we ran into was the lack of hardware available. In particular, we weren’t able to check out as many LEDs as we needed. We also needed some components, like a female DC power adapter, that were not present at the hardware booth. And so, we had limited resources and had to make do with what we had.

## Accomplishments that we're proud of

We’re really happy that we were able to create a working prototype together as a team. Some of the members on the team are also really proud of the fact that they are now able to play Ode to Joy on the violin!

## What we learned

Do not crimp lights too hard.
Things are always harder than they seem to be.
Ode to Joy on the violin :)

## What's next for Fretless

We can make the LEDs smaller and less intrusive on the violin, ideally a LED pad that covers the entire fingerboard. Also, we would like to expand the software to include more instruments, such as cello, bass, guitar, and pipa. Finally, we would like to corporate a PDF sheet music to MIDI file converter so that people can learn to play a wider range of songs.
## Inspiration

Large Language Models (LLMs) are limited by a token cap, making it difficult for them to process large contexts, such as entire codebases. We wanted to overcome this limitation and provide a solution that enables LLMs to handle extensive projects more efficiently.

## What it does

LLM Pro Max intelligently breaks a codebase into manageable chunks and feeds only the relevant information to the LLM, ensuring token efficiency and improved response accuracy. It also provides an interactive dependency graph that visualizes the relationships between different parts of the codebase, making it easier to understand complex dependencies.

## How we built it

Our landing page and chatbot interface were developed using React. We used Python and Pyvis to create an interactive visualization graph, while FastAPI powered the backend for dependency graph content. We've added third-party authentication using the GitHub Social Identity Provider on Auth0. We set up our project's backend using Convex and also added a Convex database to store the chats. We implemented Chroma for vector embeddings of GitHub codebases, leveraging advanced Retrieval-Augmented Generation (RAG) techniques, including query expansion and re-ranking. This enhanced the Cohere-powered chatbot’s ability to respond with high accuracy by focusing on relevant sections of the codebase.

## Challenges we ran into

We faced a learning curve with vector embedding codebases and applying new RAG techniques. Integrating all the components—especially since different team members worked on separate parts—posed a challenge when connecting everything at the end.

## Accomplishments that we're proud of

We successfully created a fully functional repo agent capable of retrieving and presenting highly relevant and accurate information from GitHub repositories. This feat was made possible through RAG techniques, surpassing the limits of current chatbots restricted by character context.

## What we learned

We deepened our understanding of vector embedding, enhanced our skills with RAG techniques, and gained valuable experience in team collaboration and merging diverse components into a cohesive product.

## What's next for LLM Pro Max

We aim to improve the user interface and refine the chatbot’s interactions, making the experience even smoother and more visually appealing. (Please Fund Us)
## Inspiration

Carscendo’s mission is to empower non-musical people to be creators, with a new medium for music composition. We aim to revolutionise **accessibility and fun in music creation**.

*From real-world users:*

**1. Learning music isn’t fun, getting good takes >10,000 hours:**
Our parents forced 2 members of our team to learn a musical instrument as kids. We hated it. It simply wasn’t fun. It took us a long time to reach a level high enough to compose music.
We wanted to make music composition more accessible and fun for **absolute beginners** and people who **wouldn’t normally associate music/art with themselves**.

**2. Most people can drive, few can play an instrument**:
The majority of adults are able to drive a vehicle, while only those that went through childhood toil can play an instrument – we saw some potential there. Can we use what people already know? To make music? To learn about music? To create?
Music/playing an instrument is seen as an elitist hobby, many adults stay away from it – we want to break that assumption.

**3. Dyslexic, Deaf, Blind, Paraplegia – Music Accessibility:**
How can the deaf hear music? How can the Dyslexic and Blind read sheet music? How can the physically paralysed (Paraplegia) play an instrument?
We wanted to design a new form of music information, a new medium of sense to convey what these groups of people lack.
Inspiration from a [Taste Visualization scene](https://www.youtube.com/watch?v=xizttM_Cbuc) from The movie “Ratatouille”

## Mission Statement:

**Carscendo’s is to provide a passive music compositional platform through the innovative medium of a multiplayer VR racing experience.**

#### What exactly does that entail?

Through a combination of environmental and inter-vehicular interactions, the way a player races is reflected through changes in 3 core musical elements: emphasis, tone, and rhythm. By playing Carscendo, users who may consider themselves not musically inclined and discouraged from using conventional composition systems are able to passively develop an intuition for the core principles of music composition while enjoying a multiplayer racing experience.

#### Why a racing game?

The car racing aspects not only provide an interface that people of most ages and backgrounds are familiar with as a concept (driving a car), but allow non-musicians to derive enjoyment from practice even if their musical skills are poor. By giving players this alternative outlet for enjoyment, they are encouraged to play for long enough time to develop musical intuition without even realizing it.

## Considerations

**1. Crescendo’s capabilities grow with the player’s experience level as they play the game**

As players improve their skills in the game, they gain greater agency over the music their ‘drive’ produces, meaning that new and experienced players alike have something foundational to enjoy together.

**2. Carsendo should serve as a fun experience for anyone to play and learn from- regardless of physical ability**

While the mainstream VR gaming industry is still in its early stages, we think it crucial to set the standards for inclusivity higher than they have been for conventional games.

Many games assume a high level of physical and cognitive ability, thus players who may have impairments in either of those two areas either have to exert considerably more effort than a fully abled player, not be able to enjoy many facets of the experience, or not be able to play the game at all.

In the design of Carscendo, we implemented the gameplay mechanics in as inclusive a way as possible. Some of these specific considerations include:

* visual elements paired with any audio feedback for hard of hearing or deaf players
* controls that require low levels of digit dexterity for players with tremors or joint issues
* a UI system based on environment interactions rather than text for those who struggle reading, and so it can be easily understood regardless of language.

## What it does

*How we improve accessibility and empower creativity & fun:*

**1. More Accessible and Fun medium of Music Composition: built-into racing game mechanics:**

* **Car selection** chooses a **background track.**
* Driving at **different speeds** changes the **rhythm and volume** of the background track.
* Collecting **power-ups** (like mario-kart) on the road, **adds melodies and tunes**.
* **Looping** feature allows **remixing and deeper composing.**
* **Record** feature allows **exporting of creation**.
* **Multiplayer** allows **collaboration** between experienced/inexperienced friends.

Driving is the mechanism, the proof of concept we are demonstrating – that music can be created in more modern ways with current technology than the old strings and drums. In the future, we want to explore other mediums/methods/mechanisms to make more people creators.

**2. Dyslexic, Deaf, Blind, Paraplegia – Music Accessibility:**

* **Deaf**: we represent music through stunning audio visualisations
* **Dyslexic**: we represent music notes through patterns and shapes (easier to read)
* **Blind**: we use haptic feedback and vibrations to aid sensory
* **Physically paralysed**: we use oculus quest 2 eye tracking to allow interactions of driving

## How we built it

* **VR**: Oculus Quest, Unity3D, C#
* **3D Modelling & Animation:** Autodesk Maya, Adobe Photoshop, Unity3D, Blender
* **UX & UI:** Figma, Unity2D, Unity3D
* **Graphic Design**: Adobe Photoshop, Procreate, Adobe Illustrator
* **Audio Visualisation:** Frequency analysis, Fast Fourier Transform, Koch Fractals, Audio Spectrum Analysis
* **Android version**: CAD for 3D printable steering wheel with android phone housing

## Challenges we ran into

Challenges:

* VR Interaction physics are really difficult to write from the ground up.
* Off the shelf assets vary in quality. They can be either or low or high poly and the buyer has no way of knowing before hand. Placing market assets into the scene tanked our performance and we had to find other ways to make our scene pretty.
* The design of audio visualisation is just as important as the functionality.
* The performance behind features can break whether it's even able to be included in final solution. We made 3 audio visualisations (for background track, for power-ups, for in-car), while running on a computer was fine – it was too heavy to include more than one for the headset.

## Accomplishments that we're proud of

**For the scope of this hackathon our goal was to establish the ground work for how an unconventional medium could manipulate music while simultaneously standing alone as a racing simulator that users would enjoy playing.**
We knew that creating a game of any kind in 36 hours- let alone one that had extremely innovative functionalities was an extremely ambitious task. The combination of virtual reality, real-time audio visualisation, and reactive audio based on user interactions, was a monumental goal.

## What we learned

Each member of the team utilised challenging technology, and as a result learned a lot about Unity during the last 36 hours! We learned how to design, train and test a full racing simulator and audio visualisation system in Unity and build 3D models and UI elements in VR.

This project really helped us have a better understanding of many of the capabilities within Oculus, and in utilising signal processing to create a new form of expression to use in an gamified setting. We learned so much through this project and from each other, and had a really great time working as a team!

## What's next for Carscendo

. From here some things we want to build on on in the future include

* Multiplayer Game Mode Online
* Player Map development capabilities
* More interactive musical control elements
* Environment interactions as basis for every menu action
* Real world driving - song records
* Other unconventional music platforms (how might other common tasks like cooking, etc. be used to help people learn new things?