Is there a way to translate the concept of batch size into reinforcement learning? I am using a neural network as my function approximator for reinforcement learning. In order to get it to train well, I need to choose a good learning rate. Hand-picking one is difficult, so I read up on methods of programmatically choosing a learning rate. I came across this blog post, Finding Good Learning Rate and The One Cycle Policy, about finding cyclical learning rate and finding good bounds for learning rates.
All the articles about this method talk about measuring loss across batches in the data. However, as I understand it, in Reinforcement Learning tasks do not really have any &quot;batches&quot;, they just have episodes that can be generated by an environment as many times as one wants, which also gives rewards that are then used to optimize the network.
Is there a way to translate the concept of batch size into reinforcement learning, or a way to use this method of cyclical learning rates with reinforcement learning?
Potentially.

If you do offline reinforcement learning, you're basically learning to approximate a function by sampling input/output pairs, rather than episode-by-episode. Here, your batch size could be set exactly as in an ordinary supervised learning problem.

If you do online learning, then it's not clear to me that the techniques used to set the learning rate in supervised learning can be directly applied though.

Both approaches are well covered in the RL chapter of Russell &amp; Norvig (17? 18?).