How to I specify model.learn() to end within a certain episodes of stable baselines 3?
I know specifying that total_timesteps= is a require parameter, but how to I end model.learn() within a certain episodes? Forgive me for I'm still new to stables_baselines3 and pytorch still not how to implement it in code.
import gym
import numpy as np
from stable_baselines3 import DDPG
from stable_baselines3.common.noise import NormalActionNoise

env = gym.make('NeuralTraffic-v1')

n_actions = env.action_space.shape[-1]
action_noise = NormalActionNoise(mean=np.zeros(n_actions), sigma=0.1 * np.ones(n_actions))
model = DDPG(&quot;MlpPolicy&quot;, env, action_noise=action_noise, verbose=1)
model.learn(total_timesteps=60, log_interval=1)
model.save(&quot;ddpg&quot;)
env = model.get_env()

I wanted to ended the episode on 60 instead my rollout was:
----------------------------------
| rollout/           |           |
|    ep_len_mean     | 94        |
|    ep_rew_mean     | -2.36e+04 |
| time/              |           |
|    episodes        | 1         |
|    fps             | 0         |
|    time_elapsed    | 452       |
|    total_timesteps | 94        |
----------------------------------

I don't understand why is it only 1 episode? I'd like to learn how to implement to restrict learning to specified episodes.

Generic Box-2D and classic control environments have 1000 timesteps within one episode but this is not constant as the agent can do some weird thing in the beginning and the environment can reset itself (resulting in uneven timesteps per episode). So it's the norm to keep a specific timestep in mind while benchmarking (1e6 in most research papers in model-free RL) on contrary to specifying a certain number of episodes. As you can see in SB3 Docs DDPG.learn method that they don't provide a specific argument to set the number of episodes and it is actually best to keep a specific number of timesteps in mind. I see that you have written 60 in place of total_timesteps. It's way too little to train an RL agent. Try keeping something like 1e5 or 1e6 and you might see good results. Good Luck!
