How can you represent the state and action spaces for a card game in the case of a variable number of cards and actions? I know how a machine can learn to play Atari games (Breakout): Playing Atari with Reinforcement Learning. With the same technique, it is even possible to play FPS games (Doom): Playing FPS Games with Reinforcement Learning. Further studies even investigated multiagent scenarios (Pong): Multiagent Cooperation and Competition with Deep Reinforcement Learning.
And even another awesome article for the interested user in the context of deep reinforcement learning (easy and a must-read for beginners): Demystifying Deep Reinforcement Learning.
I was thrilled by these results and immediately wanted to try them in some simple &quot;board/card game scenarios&quot;, i.e. writing AI for some simple games in order to learn more about &quot;deep learning&quot;. Of course, thinking that I can apply the techniques above easily in my scenarios was stupid. All examples above are based on convolutional nets (image recognition) and some other assumptions, which might not be applicable in my scenarios.
I have two main questions.

If you have a card game and the AI shall play a card from its hand, you could think about the cards (amongst other stuff) as the current game state. You can easily define some sort of neural net and feed it with the card data. In a trivial case, the cards are just numbered. I do not know the net type, which would be suitable, but I guess deep reinforcement learning strategies could be applied easily then.
However, I can only imagine this, if there is a constant number of hand cards. In the examples above, the number of pixels is also constant, for example. What if a player can have a different number of cards? What to do, if a player can have an infinite number of cards? Of course, this is just a theoretical question as no game has an infinite number of cards.

In the initial examples, the action space is constant. What can you do, if the action space is not? This more or less follows from my previous problem. If you have 3 cards, you can play cards 1, 2, or 3. If you have 5 cards, you can play cards 1, 2, 3, 4 or 5, etc. It is also common in card games, that it is not allowed to play a card. Could this be tackled with a negative reward?


So, which &quot;tricks&quot; can be used, e.g. always assume a constant number of cards with &quot;filling values&quot;, which is only applicable in the non-infinite case (anyways unrealistic and even humans could not play well with that)? Are there articles, which examine such things already?
Instead of having the AI learn what action to take, you can alternatively train it to judge how "good" a position is. In order to determine what move to make, you don't ask the AI "This is the current state, what move should I make", you iterate through all possible moves, and feed the the resulting state into the AI asking "How good do you think this new state is?". You then chose the move with the resulting state that the AI liked best. (Or you probably even can combine this with a traditional MinMax approach)
I'm new to this area myself, but I'd guess you would use this approach when the action space is large, and in particular when most possible actions are not a legal option in most states.