How to tell Finder where to find private key when "Connect to Server" using SSH?
Finder in macOS Sierra appears to only add the id_rsa key by default.
If you want to add other keys you either have to add them manually or alter the configuration.

on my machine a simple ssh-add ~/.ssh/test.key worked.  

According to this guide you can also store the keys in your keychain:

In ~/.ssh create config file with the following content:

Host * (asterisk for all hosts or add specific host)
  AddKeysToAgent yes
  UseKeychain yes
  IdentityFile &lt;key&gt; (e.g. ~/.ssh/userKey)


You can read more about this on the Apple Developer Site