Why do my normal maps look like trash?
First, if it isn't already, set your normal map texture to "non-color data." It doesn't work properly if it is set to color data.

If attaching the normal output to the normal input of the diffuse shader doesn't work, this node setup (courtesy of @david) will invert the normal map.