what is deep learningwhy deep learningwhy nowthe two key ideas of deep learning for computer vision--convolutional neural networks and backpropagation--were already well understood in the long shortterm memory (lstmalgorithmwhich is fundamental to deep learning for timeserieswas developed in and has barely changed since so why did deep learning only take off after what changed in these two decadesin generalthree technical forces are driving advances in machine learninghardware datasets and benchmarks algorithmic advances because the field is guided by experimental findings rather than by theoryalgorithmic advances only become possible when appropriate data and hardware are available to try new ideas (or scale up old ideasas is often the casemachine learning isn' mathematics or physicswhere major advances can be done with pen and piece of paper it' an engineering science the real bottlenecks throughout the and were data and hardware but here' what happened during that timethe internet took offand high-performance graphics chips were developed for the needs of the gaming market hardware between and off-the-shelf cpus became faster by factor of approximately , as resultnowadays it' possible to run small deep-learning models on your laptopwhereas this would have been intractable years ago but typical deep-learning models used in computer vision or speech recognition require orders of magnitude more computational power than what your laptop can deliver throughout the scompanies like nvidia and amd have been investing billions of dollars in developing fastmassively parallel chips (graphical processing units [gpus]to power the graphics of increasingly photorealistic video games-cheapsingle-purpose supercomputers designed to render complex scenes on your screen in real time this investment came to benefit the scientific community whenin nvidia launched cuda (massive clusters of cpus in various highly parallelizable applicationsbeginning with physics modeling deep neural networksconsisting mostly of many small matrix multiplicationsare also highly parallelizableand around some researchers began to write cuda implementations of neural nets--dan ciresan and alex krizhevsky were among the first see "flexiblehigh performance convolutional neural networks for image classification,proceedings of the nd international joint conference on artificial intelligence ( )www ijcai org/proceedings/ /papers pdf see "imagenet classification with deep convolutional neural networks,advances in neural information processing systems ( )licensed to