MCTS is a model-based reinforcement learning algorithm, and has been used as an effective policy improvement operator in deep reinforcement learning methods such as AlphaGo. MCTS searches the optimal sequence of state and action sequences by sequentially constructing a search tree with the initial state as the root. Each node has a state value and a number of visits, each initialized with 0. The following four steps are repeated as one cycle until a given convergence condition is satisfied.