what is deep learningusing only one or two layers of representationsthusthey weren' able to shine against more-refined shallow methods such as svms and random forests the key issue was that of gradient propagation through deep stacks of layers the feedback signal used to train neural networks would fade away as the number of layers increased this changed around - with the advent of several simple but important algorithmic improvements that allowed for better gradient propagationbetter activation functions for neural layers better weight-initialization schemesstarting with layer-wise pretrainingwhich was quickly abandoned better optimization schemessuch as rmsprop and adam only when these improvements began to allow for training models with or more layers did deep learning start to shine finallyin and even more advanced ways to help gradient propagation were discoveredsuch as batch normalizationresidual connectionsand depthwise separable convolutions today we can train from scratch models that are thousands of layers deep new wave of investment as deep learning became the new state of the art for computer vision in - and eventually for all perceptual tasksindustry leaders took note what followed was gradual wave of industry investment far beyond anything previously seen in the history of ai in right before deep learning took the spotlightthe total venture capital investment in ai was around $ millionwhich went almost entirely to practical applications of shallow machine-learning approaches by it had risen to staggering $ million dozens of startups launched in these three yearstrying to capitalize on the deep-learning hype meanwhilelarge tech companies such as googlefacebookbaiduand microsoft have invested in internal research departments in amounts that would most likely dwarf the flow of venture-capital money only few numbers have surfacedin google acquired the deep-learning startup deepmind for reported $ million--the largest acquisition of an ai company in history in baidu started deep-learning research center in silicon valleyinvesting $ million in the project the deep-learning hardware startup nervana systems was acquired by intel in for over $ million machine learning--in particulardeep learning--has become central to the product strategy of these tech giants in late google ceo sundar pichai stated"machine learning is coretransformative way by which we're rethinking how we're doing everything we're thoughtfully applying it across all our productsbe it searchadsyoutubeor play and we're in early daysbut you'll see us--in systematic way-apply machine learning in all these areas " sundar pichaialphabet earnings calloct licensed to