how to simulate 3dmax cellular map for displace like in video?
It sure is possible. I made a similar rock in a few minutes. You can check my settings from the image.



You can do this with one displacement map but for some reason I got better results with two. Put them underneath a subdivision surface modifier, and the displacement modifiers should have their strength reduced to the same number (.3-.5). The realism of the rocks comes from playing with the voronoi texture settings: size, and distance metric were most important. I found distance squared worked best. Also the color ramp can help level off any circular spots in the texture.

Hope that helps. Good luck!