How do I add a custom button next to the "Add Content" button? How can I add a button next to the + Add Content button on /admin/content, in Drupal 8?
You have to provide mymodule.links.action.yml in your module.

This is an example from core for the Add content action button from your question:

node.links.action.yml

node.add_page:
  route_name: node.add_page
  title: 'Add content'
  appears_on:
    - system.admin_content


Reference: Providing module-defined actions