## Inspiration

We are a team of engineering science students with backgrounds in mathematics, physics and computer science. A common passion for the implementation of mathematical methods in innovative computing contexts and the application of these technologies to physical phenomena motivated us to create this project [Parallel Fourier Computing].

## What it does

Our project is a Discrete Fourier Transform [DFT] algorithm implemented in JavaScript for sinusoid spectral decomposition with explicit support for parallel computing task distribution. This algorithm is called by a web page front-end that allows a user to program the frequency/periodicity of a sum of three sinusoids, see this function on a graphical figure, and to calculate and display the resultant DFT for this sinusoid. The program successfully identifies the constituent fundamental frequencies of a sum of three sinusoids by use of this DFT.

## How We built it

This project was built in parallel, with some team members working on DCL integration, web page front ends and algorithm writing. The DFT algorithm used was initially prototyped in Python before being ported over to JavaScript for integration with the DCL network. We tested the function of our algorithm from a wide range of frequencies and sampling rates within the human spectrum of hearing. All team members contributed to component integration towards the end of the project, ensuring compliance with the DCL method of task distribution.

## Challenges We ran into

Though our team has an educational background in Fourier analysis, we were unfamiliar with the workflows and utilities of parallel computing systems. We were principly concerned with (1) how we can fundamentally divide the job of computing a Discrete Fourier Transform into a set of sequentially uncoupled tasks for parallel processing, and (2) how we implement such an algorithm design in the JavaScript foundation that DCL relies on. Initially, our team struggled to define clearly independent computing tasks that we could offload to parallel processing units to speed up our algorithm. We overcame this challenge when we realized that we could produce analytic functions for any partial sum term in our series and pass these exact functions off for processing in parallel. One challenge we faced when adapting our code to the task distribution method of the DCL system was writing a work function that was entirely self-contained without a dependence on external libraries or extraneously long procedural logic. To avoid library dependency, we wrote our own procedural logic to handle the complex number arithmetic that's needed for a Discrete Fourier Transform.

## Accomplishments that We're proud of

Our team successfully wrote a Discrete Fourier Transform algorithm designed for parallel computing uses. We encoded custom complex number arithmetic operations into a self-contained JavaScript function. We have integrated our algorithm with the DCL task scheduler and built a web page front end with interactive controls to program sinusoid functions and to graph these functions and their Discrete Fourier Transforms. Our algorithm can successfully decompose a sum of sinusoids into its constituent frequency components.

## What We learned

Our team learned about some of the constraints that task distribution in a parallel computing network can have on the procedural logic used in task definitions. Not having access to external JavaScript libraries, for example, required custom encoding of complex number arithmetic operations needed to compute DFT terms. Our team also learned more about how DFTs can be used to decompose musical chords into its fundamental pitches.

## What's next for Parallel Fourier Computing

Next steps for our project in the back-end are to optimize the algorithm to decrease the computation time. On the front-end we would like to increase the utility of the application by allowing the user to play a note and have the algorithm determine the pitches used in making the note.

#### Domain.com submission

Our domain name is <http://parallelfouriercomputing.tech/>

#### Team Information

Team 3: Jordan Curnew, Benjamin Beggs, Philip Basaric
## Inspiration:

We wanted to combine our passions of art and computer science to form a product that produces some benefit to the world.

## What it does:

Our app converts measured audio readings into images through integer arrays, as well as value ranges that are assigned specific colors and shapes to be displayed on the user's screen. Our program features two audio options, the first allows the user to speak, sing, or play an instrument into the sound sensor, and the second option allows the user to upload an audio file that will automatically be played for our sensor to detect. Our code also features theme options, which are different variations of the shape and color settings. Users can chose an art theme, such as abstract, modern, or impressionist, which will each produce different images for the same audio input.

## How we built it:

Our first task was using Arduino sound sensor to detect the voltages produced by an audio file. We began this process by applying Firmata onto our Arduino so that it could be controlled using python. Then we defined our port and analog pin 2 so that we could take the voltage reading and convert them into an array of decimals.

Once we obtained the decimal values from the Arduino we used python's Pygame module to program a visual display. We used the draw attribute to correlate the drawing of certain shapes and colours to certain voltages. Then we used a for loop to iterate through the length of the array so that an image would be drawn for each value that was recorded by the Arduino.

We also decided to build a figma-based prototype to present how our app would prompt the user for inputs and display the final output.

## Challenges we ran into:

We are all beginner programmers, and we ran into a lot of information roadblocks, where we weren't sure how to approach certain aspects of our program. Some of our challenges included figuring out how to work with Arduino in python, getting the sound sensor to work, as well as learning how to work with the pygame module. A big issue we ran into was that our code functioned but would produce similar images for different audio inputs, making the program appear to function but not achieve our initial goal of producing unique outputs for each audio input.

## Accomplishments that we're proud of

We're proud that we were able to produce an output from our code. We expected to run into a lot of error messages in our initial trials, but we were capable of tackling all the logic and syntax errors that appeared by researching and using our (limited) prior knowledge from class. We are also proud that we got the Arduino board functioning as none of us had experience working with the sound sensor. Another accomplishment of ours was our figma prototype, as we were able to build a professional and fully functioning prototype of our app with no prior experience working with figma.

## What we learned

We gained a lot of technical skills throughout the hackathon, as well as interpersonal skills. We learnt how to optimize our collaboration by incorporating everyone's skill sets and dividing up tasks, which allowed us to tackle the creative, technical and communicational aspects of this challenge in a timely manner.

## What's next for Voltify

Our current prototype is a combination of many components, such as the audio processing code, the visual output code, and the front end app design. The next step would to combine them and streamline their connections. Specifically, we would want to find a way for the two code processes to work simultaneously, outputting the developing image as the audio segment plays. In the future we would also want to make our product independent of the Arduino to improve accessibility, as we know we can achieve a similar product using mobile device microphones. We would also want to refine the image development process, giving the audio more control over the final art piece. We would also want to make the drawings more artistically appealing, which would require a lot of trial and error to see what systems work best together to produce an artistic output. The use of the pygame module limited the types of shapes we could use in our drawing, so we would also like to find a module that allows a wider range of shape and line options to produce more unique art pieces.
## Inspiration

With two of our team members being musicians in the jazz and classical world since middle school, we've seen time and time again an overemphasis on enforcing a standardized approach toward teaching and assessing all student musicians. Yet we believe that with each student's musical journey being unique, so too must be the process of musical education. Currently, with an explosion of personalized education through tools like Duolingo for language acquisition and Khan Academy for academics, there's a surprising gap for personalized teaching curricula for music.

More specifically, we know that music lessons are not cheap. We wanted to build a product that would allow students to independently assess themselves and receive real-time feedback to improve their skills, especially for students who may not have access to private lessons due to financial or geographical barriers.

## What it does

Polyhymnia.ai is an application that allows users to gauge their skills with their choice of musical instrument and be given personalized sheet music based on their skill level so that they can improve at a gradual pace and be given individualized attention. It consists of two parts, first a generative section which constructs sheet music based on a Markov chain of note probabilities and another section which judges how well a student played the music in terms of pitch, rhythm and time.

## How we built it

We used Next.js and MongoDB for our front end and database. For the backend, we used a variety of music libraries (Lilypond, Midi, etc) as well as traditional AI libraries like TensorFlow.

## Challenges we ran into

Standardization of sound: minor differences in recorded frequency and expected frequency caused a hassle. There was quite a lot of difficulty in developing an algorithm which could systematically and consistently map imperfect audio recordings to musical symbols with the text of music being the chosen method of comparison.

Setting up API routes to communicate with the database and the currently logged-in user. It was the first time building a full-stack application for all three of us.

## Accomplishments that we're proud of

A key challenge we faced was developing a judging algorithm in the face of many potential sources of student mistakes, i.e. how do you compare the weights penalties such as a missed note compared to a note that was played in the wrong pitch or wrong rhythm? After deep consideration, one of our teammates realized that the Needleman-Wunsch algorithm he learned in his algorithms class for sequence alignment in DNA was surprisingly applicable for finding a method to standardize comparisons between the correct sheet music and a student's submission. We're particularly grateful to have the opportunity to apply what was previously theory in class to real world problems; seems like all those hours spent studying for class are useful after all!

We were also pleasantly surprised that our initial approach of infinite music generation with random walks through Markov chains; given some loose constraints with the key, this method allowed for unlimited possibilities for material for the student to experiment, play and assess with.

## What we learned

Since it was our first time building an application, we learned frontend with Next.js and backend development with Flask. We also learned about mathematical applications of Markov Chains and experimented with random walks. Perhaps most interestingly, spending hours translating and fiddling with the audio to notes pipeline has left us with not just a newfound appreciation for hashmaps, but also the unexpected repercussion of an ingrained map in our minds between notes and their frequencies.

## What's next for Polyhymnia.ai

We aim to expand Polyhymnia.ai beyond its current form, with the current short four-measure-token being just a proof of concept for future projects which may be longer, more complex, and able to handle harmonization. We hope to scale Polyhymnia.ai to an overall teaching assistant, able to generate personalized lesson plans in scales, exercises, and songs.