Are there OpenAI Gym continuing environments (other than inverted pendulum) and baselines? I would like to use OpenAI Gym to solve a continuing environment, that is, a problem with a single, never-ending episode (please note I don't mean a continuous environment with continuous state and actions).

The only continuing environment I found in their repository was the classic inverted pendulum problem, and I found no baseline methods (algorithms) that don't require episodic environments.

So I have two questions:


are there any continuing environments other than the inverted pendulum one?
is there an OpenAI Gym baseline method that I can use to solve the inverted pendulum problem as well as other continuing environments?
I'm not sure why you need a continuing environment, but actually you can make most (if not all) OpenAI Gym environments continuing. When you perform a step, you receive information about the next state, the reward, a termination signal and a dictionary with additional information. Simply ignore the termination signal if you want the episodes to continue indefinitely. In some cases you will need to modify a variable of the environments called $\texttt {_max_episode_steps}$, which may force the simulation to stop or to restart.

About your second question, check a resource called Spinning Up, from OpenAI too. They explain several methods and their implementation.