Why do I get an error about the modules/custom directory not existing?
There is no such thing as modules/contrib or modules/custom with the vanilla download of Drupal.

It is a convention you can use for organization. Create the folders yourself. Drupal will still find the modules in either one. The same principle applies to the themes folder. Some modules that say "download the JS plugin to /libraries" - you would have to create this folder too.

Drush, Drupal Console, and Composer all understand this, as well. If you are using Composer, you just need to inform it in your root composer.json, example:

    "installer-paths": {
        "docroot/core": [
            "type:drupal-core"
        ],
        "docroot/modules/contrib/{$name}": [
            "type:drupal-module"
        ],
        "docroot/modules/custom/{$name}": [
            "type:drupal-custom-module"
        ],
        "docroot/profiles/contrib/{$name}": [
            "type:drupal-profile"
        ],
        "docroot/profiles/custom/{$name}": [
            "type:drupal-custom-profile"
        ],
        "docroot/themes/contrib/{$name}": [
            "type:drupal-theme"
        ],
        "docroot/themes/custom/{$name}": [
            "type:drupal-custom-theme"
        ],
        "docroot/libraries": [
            "type:drupal-library"
        ],
        "drush/contrib/{$name}": [
            "type:drupal-drush"
        ]
    }


When requiring modules, profiles, or themes, Composer will then place them in the appropriate directory.