title: worker died when training rllib agent with 3D shape observation space
body: <p>When trying to transfer a 3D shape Box space to the agent in an arbitrary rllib multiagent environment, the program is crashed and given the hint message that “A worker died or was killed while executing a task by an unexpected system error, and the RayActorError: The actor died unexpectedly before finishing this task”. However, if the input space is a 1D space, the bug could be avoided. The bug is not related to the specific 3D shape sizes, shape=(84, 84, 3) or any customized 3D shape size will have the same problem.</p>
  <p>Are there any ways to input a 3D shape without this error? ray version is 1.9.2</p>
  <p>The core code is (for example use a A3C Trainer):</p>
  <pre><code>trainer = A3CTrainer(env=&quot;env&quot;, config={
      &quot;multiagent&quot;: {
          &quot;policies&quot;: {
              '0': (A3CTFPolicy, spaces.Box(low=0., high=1., shape=(84, 84, 3)), spaces.Discrete(2), {}),  #bug occurs when the shape=(84,84,3)
              #'0': (A3CTFPolicy, spaces.Box(low=0., high=1., shape=(84, )), spaces.Discrete(2), {}), #bug is avoided when shape = (84,)
          },
          &quot;policy_mapping_fn&quot;: lambda id: '0'  #unique policy
      },
  })
  </code></pre>
  <p>The log is:
  Traceback (most recent call last):</p>
  <p>File &quot;...\A3C - fortest.py&quot;, line 12, in
  trainer = A3CTrainer(env=&quot;env&quot;, config={</p>
  <p>File &quot;...\envs\tensorflow\lib\site-packages\ray\rllib\agents\trainer_template.py&quot;, line 102, in init
  Trainer.init(self, config, env, logger_creator,</p>
  <p>File &quot;...\envs\tensorflow\lib\site-packages\ray\rllib\agents\trainer.py&quot;, line 661, in init
  super().init(config, logger_creator, remote_checkpoint_dir,</p>
  <p>File &quot;...\envs\tensorflow\lib\site-packages\ray\tune\trainable.py&quot;, line 121, in init
  self.setup(copy.deepcopy(self.config))</p>
  <p>File &quot;...\envs\tensorflow\lib\site-packages\ray\rllib\agents\trainer_template.py&quot;, line 113, in setup
  super().setup(config)</p>
  <p>File &quot;...\envs\tensorflow\lib\site-packages\ray\rllib\agents\trainer.py&quot;, line 764, in setup
  self._init(self.config, self.env_creator)</p>
  <p>File &quot;...\envs\tensorflow\lib\site-packages\ray\rllib\agents\trainer_template.py&quot;, line 136, in _init
  self.workers = self._make_workers(</p>
  <p>File &quot;...\envs\tensorflow\lib\site-packages\ray\rllib\agents\trainer.py&quot;, line 1727, in _make_workers
  return WorkerSet(</p>
  <p>File &quot;...\envs\tensorflow\lib\site-packages\ray\rllib\evaluation\worker_set.py&quot;, line 87, in init
  remote_spaces = ray.get(self.remote_workers(</p>
  <p>File &quot;C:\Users\y\anaconda3\envs\tensorflow\lib\site-packages\ray_private\client_mode_hook.py&quot;, line 105, in wrapper
  return func(*args, **kwargs)
  File &quot;...\envs\tensorflow\lib\site-packages\ray\worker.py&quot;, line 1715, in get
  raise value</p>
  <p>RayActorError: The actor died unexpectedly before finishing this task.</p>
  <p>2022-02-06 11:59:07,856 WARNING worker.py:1245 -- A worker died or was killed while executing a task by an unexpected system error. To troubleshoot the problem, check the logs for the dead worker. RayTask ID: ffffffffffffffffb025425fa1d54c02ba27f8df01000000 Worker ID: 51efe6f16f87e7bc770e7c41ff526632786b8dd675e92df108c9e593 Node ID: 5c0244bcb3079a6e7ffed6081741e515f088ef511b5aad6ad7f44a60 Worker IP address: 127.0.0.1 Worker port: 36216 Worker PID: 18336
  2022-02-06 11:59:07,890 WARNING worker.py:1245 -- A worker died or was killed while executing a task by an unexpected system error. To troubleshoot the problem, check the logs for the dead worker. RayTask ID: ffffffffffffffffaa6a2753a82083400fd70c2b01000000 Worker ID: a5b0396bec32388bdf081c130a78af3ae44ee7f8372d4d2e4fefc5c8 Node ID: 5c0244bcb3079a6e7ffed6081741e515f088ef511b5aad6ad7f44a60 Worker IP address: 127.0.0.1 Worker port: 36218 Worker PID: 15244</p>
  <p>A simple complete code example to reproduce the bug:</p>
  <pre><code>import ray
  from ray.rllib.agents.a3c.a3c import A3CTrainer,A3CTFPolicy
  from ray.tune.registry import register_env
  from gym import spaces
  import numpy as np
  import gym as gym
  from ray.rllib.env.multi_agent_env import MultiAgentEnv
  
  class customEnv(MultiAgentEnv):
      def __init__(self, config):
          self.zeros = np.zeros((84, 84, 3))
          self.action_space = gym.spaces.Discrete(2)
          self.observation_space = gym.spaces.Box(0.0, 1.0, shape=(84, 84, 3), dtype=np.float32)
          self.i = 0
  
      def reset(self):
          self.i = 0
          return self.zeros
  
      def step(self, action):
          self.i += 1
          return self.zeros, 1, self.i &gt; 1000, {}
  
  if __name__ == &quot;__main__&quot;:
      register_env(&quot;env&quot;, lambda _: customEnv({}))
      ray.init()
  
      trainer = A3CTrainer(env=&quot;env&quot;, config={
          &quot;multiagent&quot;: {
              &quot;policies&quot;: {
                  '0': (A3CTFPolicy, spaces.Box(low=0., high=1., shape=(84, 84, 3)), spaces.Discrete(2), {}), 
          },
              &quot;policy_mapping_fn&quot;: lambda id: '0'  #unique policy
          },
      })
  
      for i in range(5000):
          trainer.train()
  </code></pre>