How to make the Displacement Map Texture and Material Texture Match?
You need to keep the same scale and use the same coordinates for both textures.
Alternatively you can use a mapping node to fix the scale and rotation problem.