How to get the form field label of a multi-value field
What is the proper way of obtaining the title, and what causes the
  difference?


The cause is this code in the field widget:

  // For multiple fields, title and description are handled by the wrapping
  // table.
  if ($is_multiple) {
    $element = [
      '#title' =&gt; $this-&gt;t('@title (value @number)', ['@title' =&gt; $title, '@number' =&gt; $delta + 1]),
      '#title_display' =&gt; 'invisible',
      '#description' =&gt; '',
    ];


This t($string, array $args = [], array $options = []) function returns a TranslatableMarkup object.

You can cast the translatable markup as string to obtain the title, but normally you don't have to. See both answers here  How to get the string value from the function CountryManager::getStandardList()?