Hindsight Experience Replay (HER) results obtained 50 times faster than original paper?
I am reading the results incorrectly.

1 epoch = 50 episodes = 2500 timesteps
50 epochs = 2500 episodse = 125,000 timesteps
125,000*16 CPU cores = 2 million time steps.


Will give a more detailed answer later.