Using public key with OSXFUSE
After trying some suggestions from https://github.com/osxfuse/osxfuse/issues/341 (An open issue for OSXFUSE), the following command worked for me - 

ssh-add -K ~/.ssh/id_rsa


Use of option -K:

 -K      When adding identities, each passphrase will also be stored in the user's keychain.  When
         removing identities with -d, each passphrase will be removed from it.