What is the purpose of storing the action $a$ within an experience tuple? From what I understand, experience replay works by storing tuples of $(s, a, r, s')$  to be sampled for training. I understand why we store $s$, $r$ and $s'$. However, I do not understand the need for storing the action $a$.
As I recall, the reward $r$ and the next state $s'$ are both used to calculate the target values. We can then compare these target values to the output we get when we do a forward-pass using state $s$ It seems to me that the stored action $a$ is not required for this process to work; or am I missing something? Why would we store the action $a$ if it isn't used in the training process itself?
Please, forgive me if this question has been answered before. I looked, but was unable to find anything other than generic explanations as to what experience replay is and why we do it.
We need to store the action $a$ as it tells us the action that we took in the state that we are backing up.
Suppose we are in state $s$ and we take action $a$, then we will receive a reward $r$ and next state $s'$. The goal of RL, and in particular DQN (I mention DQN as it is the first algorithm that comes to mind when I think of a replay buffer but it is of course not the only algorithm to make use of one), is that we are trying to learn optimal state-action value functions $Q(s, a)$. We thus want our value function to be able to predict $y = r + \gamma \max_{a'}Q(s', a')$, i.e. given $s$ and $a$ we want to be able to predict $y$. As you can see, we need to know which action we took in state $s$ so that we can train our value function to approximate $y$, and the value function clearly depends on $a$, hence we need to also store $a$ in our experience tuple.