Views Data Export Module hide button when unfiltered
You have a number of options, you can apply a default views filter that is applied on page load through the view settings.

You can also use javascript to test if all the filters are on "All" then hide the button with CSS