If multiple routes match a path pattern, how can I tell which one would be used? If multiple routes are returned by something like the following:
$route_provider = \Drupal::service('router.route_provider');
$found_routes = $route_provider-&gt;getRoutesByPattern('collection/%collection/items');

How can I find out which of those routes would be used if a path matched that pattern?
According to the docs:

getRoutesByPattern() returns a route collection of matching routes. The collection may be empty and will be sorted from highest to lowest fit (match of path parts) and then in ascending order by route name for routes with the same fit.

In my case, two routes with the same fitness are returned in alphabetical order:

Would the first or second one be considered more fit?
Your question is not clear enough.
Again, Your quote had full answer, the result is sorted by the fitness from highest to lowest fit. In case, they were equal, it would be sorted by alphabet of route name.
Return to your question, people usually use this function when they have a full or nearly full route name to check. or they have to add another condition that out of scope of this function.
You can not ask how Drupal determine which one wins when they are equal.