Using hook_form_alter in overlay admin theme
Thanks to Clive's tip I realized that because my module and theme had the same name, the hook was being applied at the theme level. I renamed my module and now it works!