when multiple people independently rediscovered the backpropagation algorithm- way to train chains of parametric operations using gradient-descent optimization (later in the bookwe'll precisely define these concepts)--and started applying it to neural networks the first successful practical application of neural nets came in from bell labswhen yann lecun combined the earlier ideas of convolutional neural networks and backpropagationand applied them to the problem of classifying handwritten digits the resulting networkdubbed lenetwas used by the united states postal service in the to automate the reading of zip codes on mail envelopes kernel methods as neural networks started to gain some respect among researchers in the sthanks to this first successa new approach to machine learning rose to fame and quickly sent neural nets back to oblivionkernel methods kernel methods are group of classification algorithmsthe best known of which is the support vector machine (svmthe modern formulation of an svm was developed by vladimir vapnik and corinna cortes in the early at bell labs and published in , although an older linear formulation was published by vapnik and alexey chervonenkis as early as svms aim at solving classification problems by finding good decision boundaries (see figure between two sets of points belonging to two different categories decision boundary can be thought of as line or surface separating your training data into two spaces corresponding to two categories to classify new data pointsyou just need to check which side of the decision figure decision boundary boundary they fall on svms proceed to find these boundaries in two steps the data is mapped to new high-dimensional representation where the decision boundary can be expressed as hyperplane (if the data was twodimensionalas in figure hyperplane would be straight linea good decision boundary ( separation hyperplaneis computed by trying to maximize the distance between the hyperplane and the closest data points from each classa step called maximizing the margin this allows the boundary to generalize well to new samples outside of the training dataset the technique of mapping data to high-dimensional representation where classification problem becomes simpler may look good on paperbut in practice it' often computationally intractable that' where the kernel trick comes in (the key idea that kernel methods are named afterhere' the gist of itto find good decision vladimir vapnik and corinna cortes"support-vector networks,machine learning no ( ) - vladimir vapnik and alexey chervonenkis" note on one class of perceptrons,automation and remote control ( licensed to