How to export Calibre books into Apple's Books.app?
I just drop the files, pdf or epub,  onto the open iBooks window on a mac. The "Add to Library..." menu item in the Files menu also should work. This is on 10.13.4, but earlier systems behaved the same. So if you can export your books as pdf or epub documents, they should easily transfer.