How to add a displacement map on a plane using procedural textures with a layer mask
A solution not using the Displace Modifier:

In Cycles, using the Displacement Input Socket on the material Output, you can control the displacement with a combination of texture nodes.

In this example of a simple (subdivided and UV unwrapped) plane, the wave texture is combined with a voronoi texture using an image texture of the letter A as the mix factor. In other words: the voronoi texture will only displace the parts with the letter A.





Note that this is a true 3D displacement of the geometry, not just a bump map:



As of version 2.75, this is only possible using the Experimental Feature set in the Render controls

 

and enabling True as the Displacement Method for the mesh.