Image segmentation - handcrafted features vs DNN? Currently working on a project that requires multi-class image segmentation to identify distinct anomaly types in various sheet metals.

Have had moderate success with various NN segmentation architectures, however inference performance is far from ideal given the native resolution of the images (even troublesome with extensive down-sampling).

Have also attempted to hand-craft several features based on filter banks of various convolutional kernels and then subsequently applying clustering algorithms to build a sort of 'anomaly database'. I can readily apply this defined filter bank to extract features and compared to the features to those in the labeled database. 

 

Shown in the above image, the center horizontal line needs to be identified as an anomaly. Intuition would lead me to believe simple shadow filters would be able to detect its presence, however given the inherent inconsistencies in the sheet metal - it doesn't appear to be simple (despite human identification being very simple). 

I'm not necessarily looking for a perfect solution to solve this, however if anyone was able to provide potential avenues for investigation. I want to exhaust all possible (promising) options of segmentation methods.
You have an object vs background task, i.e. two classes and instance (or m.b. class segmentation problem). If you have a labelled dataset, you can finetune one of the publicly available networks (e.g. FCN, MaskRCNN) to it to produce an image mask (i.e. an array size of the input image where every pixel is labelled as either object or background). So yeah, in 2018 feature engineering can't beat a (fully convolutional, 140M parameter, 16-layer) ConvNN.