In imitation learning, do you simply inject optimal tuples of experience $(s, a, r, s')$ into your experience replay buffer?
That seems to be functional.
That is a great approach, as long as you are using an off-policy algorithm (since the samples you are using to learn are not the policy currently being performed), like Q-learning.
By annealing the sample rate from the optimal buffer to the regular one, you introduce noise into the network and emphasize exploration (albeit more limited). This is helpful when you (the researcher) have no access to optimal policies, but merely &quot;good&quot; ones, and you still want the network to try and improve on those.