How to force regions to render in Overlay
I eventually found the code that does this in overlay.module and applied this to a custom module, using the hook:

function mymodule_system_info_alter(&amp;$info, $file, $type) {
  if ($type == 'theme') {
    $info['overlay_regions'][] = 'header';
    $info['overlay_regions'][] = 'footer';                                                                                                                                                                                                                                            
  }
}


However getting this to take effect required visiting the modules page, I don't believe simply clearing the cache worked.