Custom autocomplete field
The selection_handler element property allows to specify a plugin id of a class that would override the default selection handler for EntityAutocompleteMatcher. A separate plugin would be able to build a different Entity Query.

Create a plugin of EntityReferenceSelection type. This should be a derivative of the DefaultSelection plugin similar to TermSelection.

Then you can override entityQueryAlter to add additional conditions to the entity query.