Hindsight experience replay: strategy for sampling goals
I’ve been there for quite a while until i read the implementation of thr algorithm in OpenAI Baselines library, you can find it on Github.
Basically as of what i understood from the algorithm the K factor mainly adjusts the percentage of sampled data (being replayed for the update step in Q weights) that if we sample 1 transition without HER we sample another K with HER. This can be implemenated in the prescribed fashion if we sampled a batch of data and replayed K/1+k with HER by simply changing their goals by other states observed at any time in the future