How to make a material that toggles between 2 colors when object surface overlapping?
I'm pretty sure your example will have been done in compositing.. that leaves much more room for flexibility; you can shoot your basic plates, and then decide how to combine them at a later stage.
But if your shapes are single surfaces, and you don't mind using OSL, which means you can't use your GPU,then this could be enough for a variety of effects:
shader LayersBehind( 

 // Number of layers found behind P    
 output int LayersBehind = 0
)
{  
    point ray0 = P;     //The first of a sequence of ray origins
    int limit = 20;     //Limit on length of ray-chain
    
    while ((LayersBehind&lt;limit) &amp;&amp; trace(ray0,-I,&quot;mindist&quot;,0.000001)){
        getmessage(&quot;trace&quot;, &quot;P&quot;, ray0);
        LayersBehind++;   
    }   
}

The output of this script-node is the number of surfaces hit, behind the shading-point, on the way out from the camera.

I guess the computation for this kind of render isn't too high, so sacrificing your GPU may not be too much of a hit.