title: Rllib & Remote Desktop Connection: [Errno 10054] An existing connection was forcibly closed by the remote host
body: <p><strong>The general setting:</strong></p>
  <p>I am currently running several rllib processes on the machine at my workplace. The goal is to run multi-agent reinforcement learning simulations with a varying number of agents, differing start states and three different configurations. The computer only has the capacity to run one process at a time, which is why I loop through all configurations to allow for overnight and weekend simulations.</p>
  <p><strong>The Problem:</strong></p>
  <p>To oversee the process (e.g. if there are any problems with disk space) off work I have configured a remote desktop connection. Now after approximately 6-8 hours of running following error message is produced:</p>
  <pre><code>2022-06-08 08:01:04,008 ERROR worker.py:1259 -- listen_error_messages_raylet: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen
  2022-06-08 08:01:04,008 ERROR worker.py:488 -- print_logs: [WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen
  [2022-06-08 08:02:04,321 C 4548 1908] gcs_client.cc:343: Couldn't reconnect to GCS server. The last attempted GCS server address was 127.0.0.1:61369
  *** StackTrace Information ***
  Windows fatal exception: access violation
  </code></pre>
  <p><em>Translation: Eine vorhandene Verbindung wurde vom Remotehost geschlossen =&gt; An existing connection was forcibly closed by the remote host</em></p>
  <p><strong>The code:</strong></p>
  <p>Here is my code, even though I believe that it must have something to do with the Windows Remote Desktop connection or with the GCS server (no idea what this is or what I need it for actually):</p>
  <pre><code>def main(debug, framework=&quot;tf&quot;):
      n_agents = [8, 16, 32, 64]
      init_state = [0.1, 0.25, 0.5, 1., 2.]
      tax = [&quot;none&quot;, &quot;vote&quot;, &quot;central&quot;]
  
      for n in n_agents:
          for s in init_state:
              for t in tax:
                  shutdown()
  
                  register_env(args.env_name, lambda cnfg: ParallelPettingZooEnv(env_creator(cnfg)))
  
                  train_n_replicates = 1 if debug else 10
                  seeds = list(range(train_n_replicates))
  
                  ray.init(num_cpus=os.cpu_count(), num_gpus=0, local_mode=debug)
  
                  rllib_config, stop_config = get_rllib_config(seeds=seeds,
                                                               n_agents=n,
                                                               init_state=s,
                                                               tax=t,
                                                               debug=debug,
                                                               framework=framework)
  
                  # Define logger to use (e.g. which output formats)
                  custom_logger = LifecycleLoggerCallback(
                      logger_classes=[CSVLogger, TBXLogger],
                  )
  
                  log_dir = os.path.join(os.getcwd(), &quot;run_configurations/checkpoints&quot;)
  
                  tune_analysis = tune.run(
                      args.run,
                      config=rllib_config,
                      stop=stop_config,
                      checkpoint_freq=0,
                      checkpoint_at_end=True,
                      name=args.experiment_name,
                      local_dir=log_dir,
                      callbacks=[custom_logger],
                      trial_name_creator=trial_str_creator,
                      raise_on_failed_trial=False,
                  )
                  ray.shutdown()
  
                  for i in range(len(tune_analysis.trials)):
                      results_path = os.path.join(log_dir, args.experiment_name, str(tune_analysis.trials[i].logdir))
                      results_to_csv(results_path)
  
  
  if __name__ == &quot;__main__&quot;:
      debug_mode = False
      args = parser.parse_args()
      main(debug_mode, args.framework)
  </code></pre>
  <p>P.S.: I run the PPO RLlib-registered algorithm.</p>
  <p>Grateful for any tips. Cheers :)</p>