Views field cache tags does not invalidate the views cache
Although there are Views plugins which use CacheableDependencyInterface, I'm not sure it is the right way for a field plugin, because a field renders content and a cache tag of rendered content belongs in the render array.

This was discussed in this issue for block plugins. Don't know if this here or in the other issues for blocks is a core issues, because you can see in both core modules the effort to collect plugin cacheable metadata, or Drupal assumes, that all render arrays fulfill the requirements outlined in the docs:


  "It is of the utmost importance that you inform the Render API of the
  cacheability of a render array"


Anyways, an easy fix is to put the cache tag in the rendered output of the field.