hook_page_alter and trying to change the '#theme_wrappers' property
As I commented above, the issue was that I was missing the &amp; before $vars in this line:

$element = $vars['page']['sidebar_first'];


It was not creating a reference so it wasn't actually setting my '#theme_wrappers' property.