Limit Taxonomy Term references to special vocabulary
I see two mistake in your code, first -&gt;setSetting('target_type','taxonomy' ) is mistake and try -&gt;setSetting('target_type','taxonomy_term' )  and try to set handler_settings with  setSetting I mean  -&gt;setSetting('handler_settings',['target_bundles'=&gt;['book_cat_agency'=&gt;'book_cat_agency']] ) (although there is a way to set handler_settings in setDisplayOptions  )

and finally try  following 

$fields['book_cat'] = BaseFieldDefinition::create('entity_reference')
   -&gt;setLabel(t('Book Category'))
   -&gt;setDescription(t('Book Category Name'))
   -&gt;setSetting('target_type','taxonomy_term' )
   -&gt;setSetting('handler','default' )
   -&gt;setSetting('handler_settings',['target_bundles'=&gt;['book_cat_agency'=&gt;'book_cat_agency']] )
   -&gt;setTranslatable(TRUE)
   -&gt;setDisplayOptions('form', array(
       'type' =&gt; 'options_select',
       'weight' =&gt; -1,
       'settings' =&gt; array(
           'size' =&gt; '60',
           'placeholder' =&gt; '',
       ),
   ))
   -&gt;setDisplayConfigurable('form', TRUE)
   -&gt;setDisplayConfigurable('view', TRUE);