How can I set a 'global' Exposed filter rewrite for taxonomy terms on a view
Found a excellent module Token formatters. Can be used in various places, but for this application it's spot on.

Once installed, the formatter for a entity reference field on a view gets and addition in the dropdown 'Tokenized Text'. That in turn allow for a a wider range of token us. [see all your tokens at Admin -> Help -> Token ].

Drop the Token of your choice and swap out between fields or Taxonomies.

In my case I has an additional field on my Taxonomy terms. All the regular places I tried to reference it only gave termID or termLabel as options. Now I can use the build in features of taxonomy terms, but on one of their custom text fields.