How can I restrict libraries to attach to my custom controller only?
Your render array can return the #attached property:

$build = [
 '#title' =&gt; 'My Controller',
 '#plain_text' =&gt; 'Here is some text.'
];

$build['#attached']['library'][] = 'mycontroller/mylibrary';

return $build;


See: https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Render%21theme.api.php/group/theme_render/8.2.x