Ceiling Fan Installation - Nonstandard setup for Downrod Fan
You have to install an electrical box. What you have now is not only incorrect but it could be dangerous.

You could, depending on the access to the attic, box out a small section of the ceiling and put a regular ceiling fan box. The other option is an angled electrical box, not sure they have one that will match your ceiling slope but search for "cathedral ceiling fan box" or something similar. 

Either option will require you to secure it correctly to the framing to support a ceiling fan.

Ultimately this is what you are trying to get to: