'Add new comment' link/button does not display on custom entity
Turns out one needs to implement field formatter for other entities instead of hook_links_alter. So one can perhaps add a Pseudo field formatter to get this markup rendered on custom entities with an approach described here
The hidden gem is noted in the comments for the node entity.
  /**
   * Implements hook_node_links_alter().
   */
  function comment_node_links_alter(array &amp;$links, NodeInterface $node, array &amp;$context) {
    // Comment links are only added to node entity type for backwards
    // compatibility. Should you require comment links for other entity types you
    // can do so by implementing a new field formatter.
    // @todo Make this configurable from the formatter. See
    //   https://www.drupal.org/node/1901110.

    $comment_links = \Drupal::service('comment.link_builder')-&gt;buildCommentedEntityLinks($node, $context);
    $links += $comment_links;
  }