How do I control the visiblity of an exposed filter in a taxonomy term view?
There is an option in views to display your exposed filters in a block. In this usecase, if you only have one exposed filter, it might work for you to put the exposed filter in a block and then identify the pages you want to display it on in the block administration page. 

You could use url aliases and a wildcard * to isolate all terms of a certain vocabulary to save you listing each page in the block admin form. 

To put the exposed view in the block: Click on the advanced tab over on the right of the views pane and change 'Exposed form in block:' from No to Yes.