True displacement mid level value
Ok silly me,

It seems that the color black is used for zero displacement. So you have to use a math node on your height map to generate negative displacement with negative color values.