Checking if a machine_name exists or not
The exists key wants a callback that it can test for boolean true/false. The machine name to test will be passed as the first (and only) argument to that callback. 

Since your entity class (not the list builder class) already has a load method, inherited from Entity::load(), which accepts an ID and returns either an entity object or a null/false value, you can (and probably should) use that.

So, assuming your entity is defined as \Drupal\mymodule\Entity\MyEntity, you would use the following:

'#machine_name' =&gt; array(
  'exists' =&gt; '\Drupal\mymodule\Entity\MyEntity::load',


Exactly the same method is used in the CommentTypeForm, and probably others:

$form['id'] = array(
  '#type' =&gt; 'machine_name',
  '#default_value' =&gt; $comment_type-&gt;id(),
  '#machine_name' =&gt; array(
    'exists' =&gt; '\Drupal\comment\Entity\CommentType::load',
  ),
  '#maxlength' =&gt; EntityTypeInterface::BUNDLE_MAX_LENGTH,
  '#disabled' =&gt; !$comment_type-&gt;isNew(),
);