Client login without sending a password I would love to have a system where the password is never send in plaintext (even with TLS) to authenticate a user. I've come up with this protocol, but I'd rather use something proven.

Registration:


Client (javascript in a browser) generates a 2048 bit RSA keypair (using Forge js)
Client generates hash from the users password using pbkdf2 with 68000 rounds
Client encrypts the private key with the above hash as the passphrase using AES 128 in OpenSSH format
Client sends the encrypted private key, public key, pbkdf2 salt and the username to the server


Login:


Client requests the encrypted private key and the pbkdf2 salt using the login name
Client generates hash from the users password using pbkdf2 with 68000 rounds and salt from the server
Client decrypts the private key using the pbkdf2 hash
Client requests a challenge from the server
Server sends an challenge that is encrypted using the user's public key using OAEP-SHA1 (Using cryptography.io)
Client decrypts the challenge and sends the plaintext challenge to the server
Server sends an limited use token to the client to use the api with


TLDR: Is there a login protocol where the client can prove authenticity to a server without sending a password. The user can then use a token for API calls that expire after a set time/a few calls. This system cannot use oauth.
I can see some weaknesses in your protocol. For example, it allows any attacker to request the encrypted private key and thus mount an offline dictionary or brute force attack on the password. (An incorrect password doesn't give you a properly formatted OpenSSH key.)

Thus this is a good idea:


  I'd rather use something proven.


As mentioned in the comments SRP does exactly what you want. Here is description of the version 6(a) protocol.

To protect against compromised servers (like normal with password authentication), you'll want to incorporate your idea of using PBKDF2 (or another password hashing function). If you don't want the server to have to compute a slow password hash on each login attempt, you can use a normal hash as $H$ except when deriving $x$, which only the client needs to do. Or if you use an SRP library (as you should if possible) and it doesn't support this, you could calculate the $p$ you feed into SRP using PBKDF2 from the actual password.