How do I show a link to the user registration page only to anonymous users?
According to Structure of routes


  _role: A specific Drupal role, eg. 'administrator'. You can specify multiple ones via "," for AND and "+" for OR logic. Note that, since
  the roles available may differ between sites, it's recommended to use
  permission based access restriction when possible.


so try 

requirements:
  _role: anonymous