How to set a specific vocabulary in a taxonomy reference field during migration?
The field settings are created in the process plugin, d7_field_settings, in this section of the process pipeline.

 settings:
    plugin: d7_field_settings


It seems that you will need to write a custom process plugin. You may want to preserve the settings created by d7_field_settings. I don't recall what d7_field_settings does. So you could put something like this after the settings property is set..

 'settings/handler_settings/target_bundles': my_custom_plugin


Edit:
Glad got it working. Found the core issue for this today, https://www.drupal.org/project/drupal/issues/2763637.