How to override ViewsExposedFilterBlock with my own class?
Thanks to @4k4 (here) and @nikathone and @dorf (via Slack), I was able to arrive at this, placed in exposed_filter_images.module (all caps should be replaced with a machine name):

function exposed_filter_images_block_alter(&amp;$plugins) {
  if (isset($plugins['views_exposed_filter_block:VIEW_NAME-DISPLAY_NAME'])) {
    $plugins['views_exposed_filter_block:VIEW_NAME-DISPLAY_NAME']['class'] = \Drupal\exposed_filter_images\Plugin\Block\ExposedFilterImagesBlock::class;
  }
}