Delete font library without opening Font Book OR restore Font Book completely
You can use Pacifist to reinstall Font Book in your system. It will ask you if you want to replace it. You will need an image of your OS in an accessible pack.

Don't know if you can 'rm' the folder's content. Never tried, but fonts must be installed in the user's location and not in the System folder.

I think with 'setenv' we can link external folders with custom organized fonts to system/paths.

I'm working around because for years I used many fonts and tried to reorganize them with a minimal system set and no third-party apps like FontExplorer or others.