How to handle the final state in experience replay?
You do not store a terminal state as $s$ in the replay table because by definition its value is always $0$, and there is no action, reward or next state. There is literally nothing to learn.
However, you may find it useful to store information that $s'$ is actually a terminal state, in case this is not obvious. That is typically achieved by storing an additional done boolean component. This is useful, because it allows you to branch when calculating the TD target g:
s, a, r, next_s, done = replay_memory_sample()
if done:
  g = r
else:
  g = r + gamma * max( q(s') )