Eagerload inverse relation
UPDATE: 

in this case you should use :empty:, like this: 

{% set orphanBooks = craft.entries.section('books').relatedStories(':empty:') %}

ORIGINAL CONTENT 

According to the documentation, you can query the entries using your field handle: 


  TIP
  
  Most custom fields support element query parameters as well, named
  after the field handles.


https://docs.craftcms.com/v3/dev/element-queries/#executing-element-queries

You might use :notempty: as a negative selector, just like in this query: 

{% set products = craft.entries.section('product').productSale(':notempty:') %}  

These SE question and answers might be helpful, too: 

Get entries NOT related to categories

Return entries where field is &quot;not empty&quot;