What is the best way to represent audio visually? (x-post from UX) Original Question: https://ux.stackexchange.com/q/23040/16006

I've only taken some basic signal analysis courses, so I might be missing some things.

Purely theoretical question:

What methods exist for representing audio?

What methods could be made for representing audio, more specifically musical audio?

So far, I'm aware of:


Viewing the waveform (Soundcloud does this), mostly useless except for seeing "loudness"
Spectral analysis (Example), good for seeing frequency and "loudness"


Essentially I'm wondering if there is a way one could "see" the notes, beats, and so on of a song, visually.

Right off the top of my head I can think of displaying 3 differently colored waves over time representing treble, mid, bass in a soundcloud-like container with the section playing (or moused-over) being magnified, with the surrounding waveforms being compressed into the corners (like a wide-angle lens effect).

EDIT: I don't know where this could be used, this was just born out of my frustration with current audio visualizing technology.

I imagine having a 3d graph of a spectral analysis over time (Ninja Edit: apparently known as spectrogram) would be the "best" solution since you see everything but it might not be the most elegant and it might not be portable to places like soundcloud.

Even current spectrum analysis is hard to decipher (Too low level for images):

FL Studio wave editor

I'm essentially wondering what might work for casual users, and for people wondering ahead of time how the song will play out.
What a human (or their ear-brain) perceives in sound is a psychoacoustic phenomena, and may or may not be exactly related to the actual audio as recorded.  e.g. the exact notes, beats and instruments that a human "hears" may be influenced by visual cues, memory of other similar music, and the musical context around the actual sound of the note in question.