How can I design a reinforcement learning model for a game with multiple complex actions taken at a time?
The master's thesis Action space representation in combinatorial multi-armed bandits (2015) seems to provide an answer to my question.
Several algorithms can be used

Naive Monte-Carlo Sampling (NMC)
Linear Side Information (LSI)
Monte Carlo Tree Search with Hierarchical Expansion (MCTS-HE)
MCTS with Dimensional Expansion

The idea is to divide and conquer.
Here's a screenshot of the HE algorithm from section 4.2.