Programmatically add a taxonomy term as entity reference in a custom entity
Well thank you, the answer of 4k4 was very useful.
My problem was that for my field I wanted to reference to a taxonomy term, I had already created it and dropped it directly in the database.
It's seems no to be the right way to do this, so I was having such trouble recreating it.
So unisntalled my module (after removing all the data) , and reinstalled it , and then this piece of code worked perfectly to have a reference to a taxonomy term:

$fields['speciality'] = BaseFieldDefinition::create('entity_reference')
  -&gt;setSetting('target_type', 'taxonomy_term')
  -&gt;setSetting('handler', 'default:taxonomy_term')
  -&gt;setSetting('handler_settings', 
      array(
    'target_bundles' =&gt; array(
     'specialite' =&gt; 'specialite'
    )))
  -&gt;setDisplayOptions('view', array(
    'label' =&gt; 'hidden',
    'type' =&gt; 'author',
    'weight' =&gt; 0,
  ))
  -&gt;setDisplayOptions('form', array(
    'type' =&gt; 'entity_reference_autocomplete',
    'weight' =&gt; 3,
    'settings' =&gt; array(
      'match_operator' =&gt; 'CONTAINS',
      'size' =&gt; '10',
      'autocomplete_type' =&gt; 'tags',
      'placeholder' =&gt; '',
    ),
  ))
  -&gt;setDisplayConfigurable('form', TRUE)
  -&gt;setDisplayConfigurable('view', TRUE);


PS: specialite is my the name of my vocabulary