How to enforce a one-to-one relationship
I think you will need to use an entity reference view to enforce your restriction. In the settings for your reference type you can select Default or Views: Filter by an entity reference view

You will need to create your View first, instead of adding a Page or Block you need to add Entity Reference but this isn't available as part of the Views Wizard.

Part of the filter criteria will be to hide books which have already been assigned to a student. Hope this helps you get to your solution