How to change permission for dynamic routes?
The _permission route setting isn't a callback function like the access callback in Drupal 7; it defines one or more permissions that are checked before granting access to the route. There's nothing dynamic about the setting. Adding one (or more) 'administer taxonomy_id vocabulary terms' permissions will restrict access if the user doesn't have those one or more permissions.

If you need to define a dynamic access check, you'll need to utilize _custom_access and define your own callback function that takes into account a specific permission check based on the context (see Structure of routes for more information).