How can I get rid of the black dots or black pixels in a float 32-bit EXR baked displacement map?
In Blender, use compositor's despeckle node to get rid of those. The threshold setting is very important, this should do:



Then save the image as OpenEXR.