Hook_form_alter for changing field labels doesn't work of user registration page
Simple Solution just change ['0']['#title'] by [0]['value']['#title'].




/**
 * @param $form
 * @param \Drupal\Core\Form\FormStateInterface $form_state
 * @param $form_id
 */
function MYMODULE_form_alter(&$form, Drupal\Core\Form\FormStateInterface $form_state, $form_id){
    if ($form_id == 'user_register_form') {
        $form['field_user_first_name']['widget'][0]['value']['#title'] = 'First Name';
        $form['field_user_last_name']['widget'][0]['value']['#title'] = 'Last Name';
        $form['field_user_phone_number']['widget'][0]['value']['#title'] = 'Phone Number';
        $form['field_user_country_of_residence']['widget'][0]['value']['#title'] = 'Country (of residence)';
    }
}




This solution a little bit complicated.
After i test your code, i did some search, i found this Post say  


  It is apparent that we modify any form in drupal by implementing hook_form_alter(). But there are cases when even the hook_form_alter() cannot offer a solution in altering the form fields.  



I tried to use hook_element_info_alter as it suggested to alter fields.
In your case Here is the code bellow.
 

/**
* Implements hook_element_info_alter().
*/
function MYMODULE_element_info_alter(&amp;$type) {
  if (isset($type['textfield'])) {
    $type['textfield']['#process'][] = 'MYMODULE_process_textfield';
  }
}

/**
* Process Callback Function
*
* Callback to process the User fields.
*/
function MYMODULE_process_textfield($element) {
  //- get the current route name
  $route_name = \Drupal::routeMatch()->getRouteName();
  if ($route_name == 'user.register' &amp;&amp; isset($element['#array_parents'][0])) {
    switch ($element['#array_parents'][0]) {
      case 'field_user_first_name':
        $element['#title'] = 'New first name label';
        break;
      case 'field_user_last_name':
        $element['#title'] = 'New last name label';
        break;
      case 'field_user_phone_number':
        $element['#title'] = 'New phone numbe label';
        break;
      case 'field_user_country_of_residence':
        $element['#title'] = 'New country of residence label';
        break;
    }
  }
  return $element;
}



Note : all the four fields should be of type textfield Text (plain), otherwise you should add another process to other type, see the example in the article for password_confirm type.