Create New User Display I am looking for a way to create new user Display. 

When I go to Admin > Configuration > People > Acount Setting > Manage Dispaly

there are only three display modes: Default / Compact / User Account

How do I create a custom display mode? 

See attached picture.
In drupal 8 you can easily create a new view mode via backoffice : Administration > Structure > Display modes > View > User
and click Add new User view mode. 
To go further see documentation : https://www.drupal.org/docs/8/api/entity-api/display-modes-view-modes-and-form-modes