How do I make git credential-osxkeychain work correctly when I'm connected to a machine via SSH?
You can use ForwardAgent option in ~/.ssh/config, if it is allowed on server by option AllowAgentForwarding (in sshd_config).

Or just simply with a switch -A to ssh. It will forward connection to your agent to the server and you will be able to authenticate to github.