Trying to set up SSH with shared key, but the server is not recognizing the device's key
I did more digging, and it looks like this has been a pretty typical problem throughout the versions of cyanogenmod. Not sure if it's a bug or I just lack understanding of the problem.

I was able to connect using ssh -i and specifying the address of the key. When I had set the -v flag, it checked the key I was using and said it didn't match. So I'm thinking at best ssh ships so it's not set up correctly, and at worst, something is broken. 

I made an alias so that I don't have to type out -i then the address then the other options I want then the user and host name every time I want to connect to my home server.