How do I set a "Case Type Category" in the Compucorp CiviCase extension
There was a long discussion about the concept but the key comment is probably this one where it says it's not meant to be in the UI (or at least wasn't on the immediate roadmap) and such categories would be created by other extensions behind the scenes and specific to those extensions: https://lab.civicrm.org/dev/core/issues/630#note_16827

So it seems that yes at the moment you'd have to do it yourself outside the UI if you want your own categories.