Migrate source id's and map them to already existing Drupal entity id's
This happens to me all the time, I post a question after fiddling around for hours and shortly after, I manage to find the answer myself. It appears that the first attempt was almost right. Though bundle_key should be vid, I'm not sure why though.

The config, for completeness sake.

  field_season:
    # Map the seasons to our already existing taxonomy terms
    -
      plugin: static_map
      source: season
      map:
        '1': 26 # Lente
        '2': 29 # Zomer
        '3': 24 # Herfst
        '4': 28 # Winter
    -
      plugin: entity_lookup
      value_key: tid
      entity_type: taxonomy_term
      bundle_key: vid
      bundle: seasons