Webform 8.x-5.0-RC3 can't set custom settings
Finally, I found the answer. Webform implement ThirdPartySettingsInterface so I can edit my form_alter function to do the following : 

// Because hidden by default.
$form["third_party_settings"]["#access"] = true; 
$form['third_party_settings']['quizzes']['details_categories'] = [
    '#type' =&gt; 'details',
    '#title' =&gt; 'Catégories',
    '#open' =&gt; true,
    '#input' =&gt; false,
  ];

  $form['third_party_settings']['quizzes']['details_categories']['categories'] = [
    '#type' =&gt; 'checkboxes',
    '#options' =&gt; \Drupal\avantages\Services\PopulationsService::getPopulationList(),
    '#title' =&gt; t('Catégories'),
    '#default_value' =&gt; "",
    '#required' =&gt; false,
  ];


And to access my data, within the Webform entity : 

$webform-&gt;getThirdPartySetting("quizzes", "categories");