Category structure with multiple groups
A rather sneaky solution to your issue could be to create a matrix field that has one block type per category group. Each of those block types then could contain a single category field with the category source set to the corresponding category group. Limit those category fields to 1 category each and then limit the matrix field to one matrix block.

This way your editors will be able to select only one of the matrix block types and that block type will limit the categories to the selected group.

If you need this for some custom markup wizardry in your templates, make sure you understand variable scoping in Twig loops if you're trying to set some variable value while looping through your matrix data.