Matching a route with a "file path"
This is a case where reading the routing file is a bit deceptive.

There are actually two similar routes defined in system.routing.yml:

system.files:
  path: '/system/files/{scheme}'
  defaults:
    _controller: 'Drupal\system\FileDownloadController::download'
    scheme: private
  requirements:
    _access: 'TRUE'

system.private_file_download:
  path: '/system/files/{filepath}'
  defaults:
    _controller: 'Drupal\system\FileDownloadController::download'
  requirements:
    # Permissive regex to allow slashes in filepath see
    # http://symfony.com/doc/current/cookbook/routing/slash_in_parameter.html
    filepath: .+
    _access: 'TRUE'


After watching in the debugger, it turns out that the matched route is actually system.files, and not system.private_file_download.  But it will not match a tailed-URL at all without help.

The help needed is a "path processor", which is a routine that the router calls to let code do unusual things with route matching.  Here's the one that's used for private: and temporary: files, among other users in Core:

In core/modules/system/src/PathProcessor/PathProcessorFiles.php:

class PathProcessorFiles implements InboundPathProcessorInterface {

  /**
   * {@inheritdoc}
   */
  public function processInbound($path, Request $request) {
    if (strpos($path, '/system/files/') === 0 &amp;&amp; !$request-&gt;query-&gt;has('file')) {
      $file_path = preg_replace('|^\/system\/files\/|', '', $path);
      $request-&gt;query-&gt;set('file', $file_path);
      return '/system/files';
    }
    return $path;
  }

}


A path processor is a service, and needs a "tagged" service definition.  Here's the one for Core's path processor for the /system/files subpath:

In system.services.yml:

  path_processor.files:
    class: Drupal\system\PathProcessor\PathProcessorFiles
    tags:
      - { name: path_processor_inbound, priority: 200 }


Once I'd added the correct route and service definitions, my custom URL scheme works the same way the Core file scheme URLs work.