How to programmatically create entity with certain language applied to fields?
I have found a solution.
As I have mentioned this was during the profile installation process and it is related to How to get correct language code during installation?

The solution was to make sure the installed language exists before the modules are being installed and set it as default. Normally, translation are the last installation task. So I have copied the following code from install_download_additional_translations_operations() into a task in my profile that runs before modules are installed.

if ($langcode != 'en') {
    \Drupal::service('module_installer')-&gt;install(['language']);
    if (!($language = entity_load('configurable_language', $langcode))) {
      // Create the language if not already shipped with a profile.
      $language = ConfigurableLanguage::createFromLangcode($langcode);
    }
    $language-&gt;save();

    \Drupal::service('language.default')-&gt;set($language);
  }


I have also removed the $langcode from my code since translations were now working as expected.