Implementing experience replay in reinforcement learning
Gradient descent should be performed using the sum (or average) of the losses in the minibatch.

This is in fact also how I read the pseudocode in your question, though I understand it can be confusing. Note that, in the pseudocode, j is not specified in detail. They do not, for example, have j ranging from 0 to the size of the minibatch.

When they say:

Sample random minibatch of transitions (phi_j, a_j, r_j, phi_{j+1}) from D

they mean multiple transitions in the minibatch (with minibatch size of 1 being a special case), and they use the index j to collectively refer to the entire set of indices in that randomly sampled minibatch. It's not one particular number / index, j is a set of indices. When further lines of code do something with index j, they actually do something with all indices j.