Is there a way to limit / enforce permissions on image style downloads?
This is not really possible. The module the create this doesn't care or know where the image comes from, so you can't make any permission checks like that. It would be possible to display such images through custom theming Views and many other ways which makes a 100% failsafe permission check more or less impossible.

You can make sure nothing like this will happen based on how you setup your site, and you could add some extra security by overwriting the settings/rendering for image fields, but it won't be 100% fail safe.