Is there a query parameter to get categories that are related to anything else?
Add this to your chain...

.yourCategoryField(':notempty:')


It's not very well documented, but you can see it described in this old SE thread. In the docs, it's mentioned very briefly here.