recurrent neural networks let' look again at the equations that describe the rnn tanh ux wh yt softmax vht by applying the chain rule this we can derive the expression for is illustrated in figure -  hj  hk        hj let' now focus on the part of the expression   which involves repeated matrix multiplication of wwhich contributes to both the vanishing and exploding gradient problems intuitivelythis is similar to multiplying real valued number over and over againwhich might lead to the product shrinking to zero or exploding to infinity radient clipping simple technique to deal with exploding gradients is to rescale the norm of the gradients whenever they go over user-defined threshold and if gc then specificallyif the gradient is denoted by gw we set ggthis technique is both simple and computationally gefficientbut it does introduce an extra hyperparameter without gradient clippingthe parameters take huge descent step and flow out of the desired region with clippingthe descent step size is restricted and the parameters stay in the desired region gradient clipping will "clipthe gradientsor cap them to threshold value to prevent