Adding the category for a custom entity reference field add other fields to the same category
I fixed the problem by adding the following code to the class
  /**
   * {@inheritdoc}
   */
  public static function getPreconfiguredOptions() {
    return [];
  }