Only fetch categories that are related to a minimum of entries
What you can always do in situations, where the default parameters are not enough,
is to "collect" element IDs and pass them to the id param of a new criteria model.

So loop through all categories and merge the category IDs matching you condition to an array:

{% set topCategoriesIds = mainCategoriesIds|merge([category.id]) %}


When you're done, get your filtered categories from these IDs:

{% set categories = craft.categories.id(topCategoriesIds) %}