Create a custom fieldset template
After looking a little deeper in the Drupal core I found out that in this case I shouldn't overwrite #theme, but use #theme_wrappers instead.
So my theme definition:
function mymodule_theme($existing, $type, $theme, $path) {
  return [
    'my_custom_fieldset' =&gt; [
      'render element' =&gt; 'children',
    ],
  ];
}

In my form:
$form['some_fieldset'] = [
  '#type'  =&gt; 'fieldset',
  '#title' =&gt; 'Some title',
  '#theme_wrappers' =&gt; ['my_custom_fieldset'],
];

$form['some_fieldset']['some_field1'] = [
  '#type' =&gt; 'textfield',
  '#title' =&gt; 'Some title',
];

$form['some_fieldset']['some_field2'] = [
  '#type' =&gt; 'textfield',
  '#title' =&gt; 'Another title',
];

And then my template my-custom-fieldset.html.twig:
&lt;fieldset&gt;{{children}}&lt;/fieldset&gt;

Or for example:
&lt;fieldset&gt;
  {{ children.some_field1 }}
  Some customization here
  {{ children.some_field2 }}
  Something custom here
&lt;/fieldset&gt;