Views Exposed Filter Sort on Entity based on Term Reference Field from Entity Reference Relationship
So you have two relationships. One to the referenced entity to pull it in, and another on the term on that referenced entity to pull in data such as term names (to sort on term names).

This immediately brings to my mind an issue whereby you will have what seems like duplicate entries coming in. To avoid that you will have to use aggregation, and in the aggregation settings you will have to do a count on one of the fields (which will collapse the result set) and then do an exposed filter.