Are there conditions where invalidated cache items are returned?
See my presentation on Drupal 8 Caching about that topic: https://md-systems.github.io/drupal-8-caching/#/6/6 for how you can still get invalid items. And https://md-systems.github.io/drupal-8-caching/#/6/5 on how cache tag invalidation works.

The reason why cache tags invalidate instead of delete is scalability. Invalidating a cache tag does not require us to touch each single cache entry, we just need a single update query to increase the invalidation counter that is checked when cache entries are loaded. Common cache tags like renderered, image style cache tags and other frequently used cache tags could be on tens or hundreds of thousands of cache entries, deleting them all at once would take a very long time on MySQL and other cache backends like Memcache have no API to do do something like that at all.