macOS 10.12 Sierra will not forget my ssh keyfile passphrase
To disable storing passphrase in keychain you should add following lines to ~/.ssh/config file:

Host *
UseKeyChain no


If you already have passphrase stored in keychain you can delete it using commands:

cd ~/Library/Keychains/&lt;UUID&gt;/
sqlite3 keychain-2.db 'delete from genp where agrp="com.apple.ssh.passphrases"'


If you would like to have a behavior like in OS X El Capitan or earlier (i.e. passphrase stored until logout or restart) simply add to ~/.ssh/config:

AddKeysToAgent yes


You don't need to start ssh-agent.