How does Hindsight Experience Replay learn from unsuccessful trajectories?
Ignoring HER for now the $Q$ and $V$ functions operate on states and actions which are part of a Markov decision process which we call $M_0$.
Back to HER, the $Q$ and $V$ functions now take a goal as an additional parameter called the goal. We will denote individual goals $g_n$, the true goal $g_0$, and the set of all goals $G$. The set of goals is chosen such that every state matches at least one goal. We create a new MDP $M_1 = M_0 \times G$ (i.e. a larger MDP composed of multiple copies of $M_0$, all the states in each being tagged with a goal). The reward is either +1 or 0 depending on whether or not the goal and the component from $M_0$ match in some predefined sense. In HER trajectories are collected from the subset of $M_1$ where $g = g_0$ and added to the replay buffer. When training the $Q$ and $V$ functions we don't only use the original trajectories: we create new ones by substituting new values of $g$, which we do so strategically so as to include some trajectories with a positive reward.
Things to note:

HER doesn't assign rewards to states or trajectories in $M_0$: the reward function used is only defined for $M_1$
The performance of HER depends on $Q$ and $V$ both being models with some ability to extrapolate to unseen data points; such as neural networks, support vector machines, etc. It would not provide any benefit if applied to value tables.