Android SSH with AWS and JuiceSSH
This was primarily my own mistake and I am leaving it here only for future visitors who may face the same issue.  

I was using a wrong pem file and since the issue was not highlighted as "wrong pem" but as it needed password I was confused.