How to disable SSH passphrase keychain saves in OS X Sierra
There is a new option UseKeychain in Sierra, which defaults to yes and which is a reason for this behavior. If you do not like that, edit the configuration in ~/.ssh/config (or create this file):

UseKeychain no
AddKeysToAgent yes


The second option will tell the SSH agent to store the keys, which has the same result as if you had used ssh-add to enter your passphrase beforehand.