Explanation behind the calculation of training loss in deep learning model I am trying to model an image classification problem using convolution neural network. I came across a code on Github in which I am not able to understand the meaning of following line for loss calculation in the training loop.
I am omitting most of the detail and only placing the relevent code-

for batch_idx, (data, target) in enumerate(final_train_loader):
     loss = criterion(output,target)
     #Idea behind the below line
     train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))


Cross-entropy loss function is being used here.
The line you're asking about

train_loss = train_loss + ((1 / (batch_idx + 1)) * (loss.data - train_loss))


is basically calculating the average train_loss for the finished batches

To illustrate, suppose 4 batches have been done (with average loss named avg_loss) and current is calculated from 5th batch (with loss named new_loss) 

The new average loss is from 

$\frac {4 \times \text{avg_loss} + \text{new_loss}} {5}$

This is exactly the same as 

$\text{avg_loss} + \frac {\text{new_loss} - \text{avg_loss}} {5}$ 

which is the calculation done by the code