Minimum chainstay/crankset clearance
Answering my own question with some observations I gathered from other forums:


The bolt boss (the knurled metal ring) in the picture is visible only because the chainring was installed incorrectly. When I tightened the bolts properly, the metal boss settled inside the crank spider and I gained at least 1mm clearance. New clearance in the following picture:


 


Both chainstay and crank arms are objects that don't flex much, unless they are of poor quality, but this is not the case for my build.
The biggest risk is the chain dropping and getting trapped in that small gap. In my case the gap is small enough that it shouldn't happen. Moreover, given that it's a fixed gear setup without front mech, the chance of chain dropping are very low.  


Source: https://www.reddit.com/r/FixedGearBicycle/comments/4ck3h9/crank_arm_spider_and_chainstay_clearance_should_i/