Minimizing cache clearing during development
Define all caches disabled, how do you do that? 

The recommended approach is to only disable the render cache (as per the example local settings.php), but if you have a complex site and a slow, especially virtualized environmment, that can still take some seconds on every request.

Make sure you don't have xdebug or so constantly enabled. Use PHP7 if possible, that's basically 50% faster. It doesn't make mysql faster obviously, but unless you have very slow/bad queries, that is likely a smaller part of the whole response than you think. And if you have slow queries, then you need to find ways to optimize them, since it will get much morse when you have more parallel page requests.

If you clear cache manually, make sure you use drush cc render and not drush cr unless you actually change something that requires a full cache clear.

Cache tag invalidation doesn't really help that much, you could of course find out the cache tag of your specific block/view and invalidate just that, but there is no drush command for that yet). You could write your own cache backend that extends from the default and allows to ignore cache writes of cache IDs that match a certain pattern. But invalidation will require you to figure out the cache id of the thing, unless you work on the same thing for quite some time, that won't be worth the effort.

For more specific things, you'd need to profile the site to find out what is actually slow.