How do I use custom user forms?
There is no option for this in ui.

You have to define the display mode for the form in mymodule.routing.yml:

Example from user.routing.yml

user.admin_create:
  path: '/admin/people/create'
  defaults:
    _entity_form: 'user.register'
    _title: 'Add user'
  requirements:
    _permission: 'administer users'


In the line  _entity_form: 'user.register' the form is user and the form mode is register.

Then you can access the form with the path defined here and restrict the access through roles with requirements.

Edit (place the form on any page)

If you want to place the form on any page, you need the form as a block.

You can use this module:

https://www.drupal.org/project/entityform_block

But it does not have the option for arguments like form mode.

This experimental module can do this:

https://www.drupal.org/project/entity_forms_in_blocks


  
  Interesting features that does not exists for now with the other modules : ability to choose the form mode as Drupal 8 allows to (in
  Drupal 8, you can create additional form modes).