HIde taxonomy menu based on count? Possibly using rules?
There is an option in the taxonomy_menu "Hide empty items" to disable menu options if there are no nodes tagged with the specific term. The option can be found in the admin/structure/taxonomy then the vacab you are using.

Below is an image for reference.

Settings Image