What is the correct way to cache blocks by language?
You should use the languages context, don't use tags here.

Like this: '#cache' =&gt; ['contexts' =&gt; ['languages:language_interface']],

You'll find more here: https://www.drupal.org/developing/api/8/cache/contexts