Drupal 8: Template suggestion for form-element in a views exposed filter is not working
Ok, looks like I was too tired. The problem was that the template should be called form-element--custom.html.twig.

Notice the '--' (double dash) instead of the __ (double underscore). 

I also ended up using the more specific hook alter:

hook_theme_suggestions_HOOK_alter


...where HOOK for me was form_element.