How do I make my displacements go out of the mesh rather than inside the mesh?
This can be caused either because your normals are flipped (which is not the case I suppose) or because you auto-unwrapped the model and now you need to change the scale on your UVs (specifically on the face which looks wrong) and on the proper axis (like scaling on X by -1). 
Now for the displacement map, usually a displacement map has one channel (grayscale) which means that if you invert it (add -1 or introduce invert node or remap it with ramp) it should look towards the opposite direction. 
Hope that these were helpful! 

PS: and as you found out and mentioned on your comment "midlevel" property should do the job if you set it on a negative value (like -0.5 or -1)