How to have a field displayed depending on user's role in Views?
If this is a custom entity you may consider overriding the checkFieldAccess method of Drupal\Core\Entity\EntityAccessControlHandler.

Then assign your own handler as handlers/access in @ContentEntityType.

But it's only me who prefers being in coding hell instead of configuration hell :-)