LSTM loss function and backpropagation
For sake of convenience, I'll go in ascending order of how the neural
network is working


The forward propagation of the LSTM cell is where the input is fed  ->  passed through the hidden states  ->  output is achieved (either at each time-step or at the end depending upon the type of problem
like classification, forecasting, etc.,)
Then after receiving the output the error between the actual value and the predicted value is calculated. This is where the loss
function comes into the picture
Classification problem - cross-entropy/log-likelihood
Regression Problem - Mean Squared Error, Mean Absolute Error functions are used.
So, after calculating the error only we backpropagate through time (BPTT) in case of RNN neural networks which updates weights of the
matrices and cell states of the LSTM cells. In practice, we refer to
these backpropagation algorithms as optimization algorithms like
Gradient Descent, etc., 
The optimization algorithms like RMSProp, Adam are much faster in practice than the standard gradient descent algorithm.


Overall, the training part of any neural network algorithm is that at
the early stages of training, it will perform very horribly and over
a period of time, with the correct loss function and backpropagation
algorithm it will try to reduce the chances of classifying error and
will converge fast depending upon the backpropagation algorithm used.