Title: How to define action space in custom gym environment that receives 3 scalers and a matrix each turn?
Tags: python;reinforcement-learning;openai-gym
Question: for a personal project, I need to define a custom gym environment that runs a certain board game. each turn of the game, the environment takes the state of the board as a matrix of ones and zeros, and an action - described as a tuple:
(integer, integer, small matrix)
from reading online, I know that a gym env should take the shape:
class CustomEnv(gym.Env):
&quot;&quot;&quot;Custom Environment that follows gym interface&quot;&quot;&quot;
metadata = {'render.modes': ['human']}
def __init__(self, arg1, arg2, ...):
super(CustomEnv, self).__init__()
self.action_space =
self.observation_space =
def step(self, action):
...
def reset(self):
...
def render(self, mode='human', close=False):
now, I feel like the action input here does not exactly fall into &quot;discrete&quot; or &quot;continuous&quot; - how should I implement the action part of the init function and the step function?


Here is another answer: Defining your action space in the init function is fairly straight forward using gym's Tuple space:
from gym import spaces
space = spaces.Tuple((
spaces.Discrete(5),
spaces.Discrete(4),
spaces.Box(low=0, high=1, shape=(2, 2))))
The Discrete space represents a range of integers and the Box space to represents a n-dimensional array. You can print a sample of your space to get an idea of what it looks like:
print(space.sample())
&gt;&gt;&gt; (3, 1, array([[0.20318432, 0.26787955], [0.5323673 , 0.6564413 ]], dtype=float32))
For the step function, you just need to interact with your environment based on the input action, which will be formatted just like the sample.

{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 1, 'CommentCount': 0, 'ContentLicense': 'CC BY-SA 4.0', 'CreationDate': '2020:00:09/20/20', 'FavoriteCount': 1.0, 'PostTypeId': 1, 'Tags': '<python><reinforcement-learning><openai-gym>', 'Title': 'How to define action space in custom gym environment that receives 3 scalers and a matrix each turn?', 'ViewCount': 2720, 'id': 63977420, 'user_AboutMe': '', 'user_AccountId': 10841131.0, 'user_DisplayName': 'Moran Reznik', 'user_Reputation': 961.0, 'user_Views': 163.0, 'user_id': 7972317.0}}