incorrect passphrase
I'm assuming you've already tried caps lock, the intermediate passwords you used etc.

Ideally you would backup your wallet before making changes such that you have a library of them: wallet.dat.1jun2018, wallet.dat.2jul2018 etc. Then if something goes wrong, you already know that the previous copy of the wallet is still available.

I can't tell what's gone wrong for you from your description above, but OSX Mojave does offer some facilities. First, you MUST copy your wallet as it currently stands - even though you don't have the password - to an external disk, date it as above. Then use Time machine even if you haven't been using it to have a look at the folder ~/Library/Application Support/Bitcoin. Mojave stores local backups on your computer "even when your Time Machine backup disk isn't available." You're hoping to see that wallet.dat is available from a date prior to the encryption. 

If it is, then you might try restoring it. Also copy it under a new name to your external disk. You might be lucky enough that bitcoin-qt works straight away without the password. If you can restore a previous version but cannot seem to use it, things start to get a lot more tricky, very quickly.

If you don't have the local backups, or the wallet isn't within it, I would turn the computer off and sleep on it. When you get up, write down everything you did, every password you tried, in the hope that something jogs your memory.