occur in the process firstthe gradient of the error function may become very smallandsince backpropagation relies on multiplying these small weights togetherthe weights of the network may update very slowly or not at allpreventing proper training of the network this is known as the vanishing gradient problem converselythe other issue is that the gradient of the error function might become very largewith backpropthe weights throughout the network may update in huge incrementsmaking the training of the network very unstable this is known as the exploding gradient problem unsupervised pretraining to address these difficulties in training very deepmultilayered neural networksmachine learning researchers train neural networks in multiplesuccessive stageswhere each stage involves shallow neural network the output of one shallow network is then used as the input of the next neural network typicallythe first shallow neural network in this pipeline involves an unsupervised neural networkbut the later networks are supervised this unsupervised portion is known as greedy layer-wise unsupervised pretraining in geoffrey hinton demonstrated the successful application of unsupervised pretraining to initialize the training of deeper neural network pipelineskicking off the current deep learning revolution unsupervised pretaining allows the ai to capture an improved representation of the original input datawhich the supervised portion then takes advantage of to solve the specific task at hand this approach is called "greedybecause each portion of the neural network is trained independentlynot jointly "layer-wiserefers to the layers of the network in most modern neural networkspretraining is usually not necessary insteadall the layers are trained jointly using backpropagation major computer advances have made the vanishing gradient problem and the exploding gradient problem much more manageable unsupervised pretraining not only makes supervised problems easier to solve but also facilitates transfer learning transfer learning involves using machine learning algorithms to store knowledge gained from solving one task to solve another related task much more quickly and with considerably less data backpropagation (also known as backward propagation of errorsis gradient descent-based algorithm used by neural networks to update weights in backpropthe weights of the final layer are calculated first and then used to update the weights of the preceding layers this process continues until the weights of the very first layer are updated closer look at unsupervised algorithms