What is the difference between batches in deep Q learning and supervised learning?
From what I understood in a classifier a common method is that you sample a mini-batch, calculate the loss for every example, calculate the average loss over the whole batch and adjust the weights w.r.t the average loss? (Please correct me if I'm wrong)

You are wrong.
The weights are adjusted w.r.t. the average gradient, and this must be calculated using individual loss function results. The average loss (or cost function when considering the whole dataset) is a useful metric for current performance, and it is the measure being minimised. But you cannot calculate meaningful gradients against the average loss directly.

But is this the same in DQNs?

The batch process is not as you described, but an experience replay minibatch in RL and a sampled minibatch in supervised learning can be very similar. The main difference in RL is that your prediction targets must be recalculated as part of the sampling process (using $G_{t:t+1} = R_{t+1} + \gamma \text{max}_{a'}\hat{q}(S_{t+1},a', \theta)$ to calculate the TD target, assuming you are using single step Q learning), whilst in most supervised learning the target values are fixed for each example.
In theory you could use repeated single item stochastic gradient descent in DQN, it doesn't break any theory, and it would work. However, it will usually be more efficient to use a standard minibatch update, combining all gradients into one average gradient for the minibatch and making a single update step.
If you are using a high level library for your neural network model in DQN, you usually don't need to worry about this detail. You can use the .fit function or whatever the library provides. In that case the only difference between a supervised learning update and an experience replay DQN update is what you get from the sampling. In supervised learning you get a set of $(\mathbf{x}_i, \mathbf{y}_i)$ examples directly by sampling a minibatch. In RL you get $(\mathbf{s}_i, \mathbf{a}_i, r, \mathbf{s'}_i, done)$ and must construct the $(\mathbf{x}_i, \mathbf{y}_i)$ minibatch from these before passing to your .fit function