Converting raster shape/blob into displacement map I am a beginner in digital image processing and computer graphics. I would like to program a similar behavior than the Shadermap 3 normal editor (displacement layer more specifically).

As shown in this Shadermap 3 demonstration video, the displacement brush allows the user to paint a displacement map (grayscale)



The brush strokes produce shapes with a smooth value transitioning from the middle of the shape to its edge, in this case giving the shape a slightly curved profile. As it is also shown, the shape's profile can be fine-tuned with the brush curve editor.

What combination of filters and/or algorithms could Shadermap 3 possibly use to convert a pixel blob or shape to displacement map with a given width and curve?
One way I can think of is to make a "signed distance transform" of the image where there is information for each pixel about how far the pixel is to the closest surface of the shape.  Since it's signed, youll be able to know if the pixel is inside or outside he shape, and by how much.

Using this knowledge, you could easily make a new image, where the pixel is black if the distance is greater than -10, else it is white.  That would make the shape shrink by 10 pixels.

You can use smoothstep or other curves to transition between black and white over a number of pixels, to soften it anti alias the edges.

You can make the distance transform by brute force, but another method would be to make a voronoi diagram first and then turn that into a distance transform.

There is lots of info out there on both voronoi diagrams as well as (signed) distance transforms but these two links will probably be helpful to get you started:

http://blog.demofox.org/2016/02/29/fast-voronoi-diagrams-and-distance-dield-textures-on-the-gpu-with-the-jump-flooding-algorithm/

http://blog.demofox.org/2016/03/02/actually-making-signed-distance-field-textures-with-jfa/