How do I remove an ssh private key from ssh-agent/keychain
After I post the question, I found the answer, the following command will generate the public key using the private key file:
ssh-keygen -y -f ./MyPrivateKey.pem &gt; ./MyPublicKey.pub

then to remove it:
ssh-add -d &quot;MyPublicKey.pub&quot;

Reference