6883e621_ws___Machine_Learning__Theory___3
[0] Convex Loss Optimization [1] Specify a loss function related to the world-imposed loss fucntion which is convex on some parametric predictive system. Optimize the parametric predictive system to find the global optima. [2] Mathematically clean solutions where computational tractability is partly taken into account. Relatively automatable. [3] 
The temptation to forget that the world imposes nonconvex loss functions is sometimes overwhelming, and the mismatch is always dangerous. Limited models. Although switching to a convex loss means that some optimizations become convex, optimization on representations which aren’t single layer linear combinations is often difficult.
All Models of Learning have Flaws « Machine Learning (Theory)
3