Why isn't my implementation of DQN using TensorFlow on the FrozenWorld environment working?
I see at least 3 issues with your DQN code that need to be fixed:


You should not have separate replay memories for successes/failures. Put all of your experiences in one replay memory and sample from it uniformly.
Your replay memory is extremely small with only 2,000 samples. You need to make it significantly larger; try at least 100,000 up to 1,000,000 samples.
Your batch_target is incorrect. You need to train on returns and not just rewards. In your train function, compute the 1-step return $r + \gamma \cdot max_{a'} Q(s',a')$, remembering to set $max_{a'} Q(s',a') = 0$ if $s'$ is terminal, and then pass it to model.fit() as your prediction target.