Deep Q-Learning: why don't we use mini-batches during experience reply?
DQN "library" implementations that I have seen do use mini-batches to train, and I would generally recommend this, as it usually strikes a reasonable balance between number of weight updates and accuracy of the gradients.

In your first link, and the code excerpt, the sample list is literally called minibatch. However, the developer then goes on to make a series of individual SGD steps per item within that sample. 


  Why can't they use mini batches instead? 


They can, and it works just fine. In my DQN explorations I tend to use a single mini-batch per timestep, and the Adam optimiser to accelerate learning. I also use a mini-batch to calculate TD target values.


  I'm new to RL, but in deep learning people tends to use mini-batches as they would result in a more stable gradient. Doesn't the same principle apply to RL problems? 


It does, but I don't know if this is the most sample efficient that it could be. I prefer to use mini-batches and more complex optimisers because this runs faster (for a given number of replay training samples per time step), and I can run more time steps of the environment in less time.

It is possible for instance that a replay sample size of 32 per time step, with 4 mini-batch updates of 8 each would be the most sample efficient approach for a specific problem (because it makes more updates from the given samples), but it might not be the most CPU efficient.


  Is the randomness/noise introduced actually beneficial to the learning process? 


I don't think so, but there are plenty of other tuning issues to consider in RL.


  Am I missing something, or are these sources all wrong?


They are not necessarily wrong, but may be CPU inefficient. 

More likely in my opinion, your sources are explaining algorithms in principle, and did not want to add too many layers of optimisation, or too many components that may need tuning to work. There is a lot going on in a typical DQN agent, and having many performance optimisations can hide the details that the person is trying to teach.