How to organize content via taxonomy, blocks, panels, views?
This is a very broad question Brian, but I think it boils down to 'how should I handle relationships between entities'.

Take a look at the Entity Reference module.  With that you can create links between your various nodes so that you can display them on the same page.

For instance let's say your main content type is exercise, on each exercise node you could use the module to create links to a video and a fingering diagram (or several).  Then for each other thing you wanted to display on the page, you create a view with a contextual filter that looks for the node id of the exercise which is being displayed.

I highly recommend finding some reading or videos on how contextual filters in views works.

Panels makes the handling of contexts a little bit easier in the long run.  There's an excellent tutorial on Page Manager.