lilypond-user
[Top][All Lists]

## Re: Animated SVG Percussion Music

 From: Tim Sawyer Subject: Re: Animated SVG Percussion Music Date: Fri, 30 Sep 2011 09:20:57 +0100 User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110922 Thunderbird/3.1.15

On 30/09/11 07:12, Jay Anderson wrote:

On Fri, Aug 19, 2011 at 5:30 AM, Tim Sawyer

http://percussion360.com/

This works by taking SVG generated from lilypond, munging it a bit
(including matching it up with the .ly source), then playing it back using
javascript and html5 audio.  Playback doesn't work in IE but does work in
other recent (proper) browsers.


This is really neat. I'm thinking of doing something similar with some
hymns. How did you add the id attributes to the paths to the svg
source? I assume this is what you meant by matching it up with the
source. I'm not seeing a good way to do this yet.


It's not nice...took me quite a while to do.


I have my books split into 32 parts. Each part is in a separate .tex file. I have a python script that extracts lilypond scores from these .tex files. Once it has a score, it writes it to disk as a .ly file.


Next, we take the .ly file and convert it into an svg/xml file using "lilypond -dbackend=svg file.ly". This uses lilypond 2.13.11.


We also at this point generate a .eps file from the score, "lilypond -dbackend=eps file.ly", then use http://www.imagemagick.org to convert this into a trimmed .jpg of the score. (I made this bit optional in the generation - it's what takes the longest time)


I then create an object which wrappers the .ly file and the .svg file, and links them together. I'm parsing the music line from my .ly file so that I know what notes are involved in what order. We first work out a music resolution (ie shortest note required to accurately playback the score), and then we produce a python dictionary that has positions in the stave (in units of the music resolution) pointing to objects that hold details of the note. If it's a chord (ie snare drum and bass drum together) the note holds a chordList for the other notes.


On the SVG side, we look through the svg file produced, looking for known shapes (ie a tag with attribute d="M217 139c67 0 112 -38 112 -94c0 -90 -111 -184 -217 -184c-67 0 -112 38 -112 94c0 90 111 184 217 184z" is a solid notehead). We order this structure by position (from the value in the transform attribute) and then match up the notes between the svg and the lilypond object structure. If we get a different number of notes, then we have a problem somewhere!


We then write out a new SVG file with some additional attributes on the nodes - an id for position, a delay for length and an audio for sounds at playback. Rests are just notes with no sound.


Note that this only currently works for one line staves - I've only done this for my short exercises. I'd have to expand this matching code to cope with multiline stuff, and there's no point for what I'm doing with it.

So, here's the original .ly for my first stave:

#(set-global-staff-size 18)
\paper
{
ragged-right=##f
}
\score
{
\version "2.10.0"
{
\time 4/4
\override Staff.VerticalAxisGroup #'minimum-Y-extent = #'(-6 . 7)
\override Staff.TimeSignature #'style = #'()
\clef bass
\stemUp
e e e r e r e r r e e r e e e r \bar "|."
}
}

The lilypond generated SVG is attached (1.1.0.1.svg)


The resulting SVG that ends up in the web page is also attached (1.1.0.1.svg.xml)

This is basically the same SVG as lilypond produced with the addition of:

* audio - path to the sound to play
* id - the position in the music, in units of music resolution
* delay - number of music resolution units to sound this note for


The javascript then looks at the length of the music (see bottom of .svg.xml), grabs hold of all the elements with an id of n1 up to n16 (or the length), loads the appropriate sounds, then starts playback, setting the colour of elements to red for the specified length of time, then back to black.

Hope that helps!

Tim.


1.1.0.1.svg
Description: image/svg

1.1.0.1.svg.xml
Description: Text Data
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 1, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.401604562997818, "perplexity": 4067.2470565617564}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2014-23/segments/1406510271654.40/warc/CC-MAIN-20140728011751-00204-ip-10-146-231-18.ec2.internal.warc.gz"}