Custom template for media
Credit for most of this answer to this article: http://forumone.com/insights/styling-inline-images-with-drupals-wysiwyg-and-media-modules/
First I went and edited the relevant displays and added a caption field (which is what I was trying to do)

Then used hook_media_wysiwyg_token_to_markup_alter() to change the output, in my case I needed to catch the situation where the image got floated in the WYSIWYG, and float the parent element rather.

&lt;?php
function MODULENAME_media_wysiwyg_token_to_markup_alter(&amp;$element, $tag_info, $settings) {
    if (($element['content']['#bundle'] == 'image')) {
        // check if a caption has been set
        if(!empty($element['content']['field_caption'])){
            if(isset($element['content']['file']['#attributes']['style']) &amp;&amp; strstr($element['content']['file']['#attributes']['style'], 'float')){
                // move floats to the wrapping element
                $element['content']['#attributes']['style'] = $element['content']['file']['#attributes']['style'];
                // remove inline style on image
                $element['content']['file']['#item']['attributes']['style'] = '';
                // add class to wrapping element so we can style it
                $element['content']['#attributes']['class'][] = 'media-with-caption';
            }
        }
    }
}