In reinforcement learning multi-step decision making, each decision making step is associated with a reward/penalty, whilst the whole multi-step decision making process is associated with an expected return/cost, which is the acummulation of rewards/penalties of all decision-making steps. For each of the decision making steps, having multiple options, a well-trained policy model predicts expected returns/costs of the options by foreseeing the cumulative return/cost of the following steps from the current options, and then selects the option with maximised expected return, or the minimised expected cost. In the synthetic problem, each decision making step is to select a reaction option from the reactions pool for the current molecule, and the whole multi-step decision making process is to assemble the multi-step reaction pathway. Here to assess the reaction options, we chose the penalty-expected cost evaluation system and designed reaction assessment scores to represent the penalty to select the candidate reactions. The scores would later be also used to quantify performance of synthesis planning from the three different reactions pools.