How to make view mode available only to selected bundle? I have three taxonomy vocabularies which means that I have three bundles of the vocabulary entity type. I have a few view modes for them and one of them has to be available only to one of those vocabularies.

The method responsible for getting available view modes is \Drupal\Core\Entity\EntityManager::getAllDisplayModesByEntityType() but it does not distinct between bundles, only entity types.

Is there a way I can make that view mode restricted for that one particular vocabulary or this is just not possible?
No, there is not. A view mode is not bundle-specific.

That hardly matters, because a view mode doesn't do anything unless you actually enable it and configure field formatters for it, resulting in a view display. If you don't then it just falls back to the default view display.