In general, the RL-CSP algorithm updates the policy after a certain sequence of steps, called an episode, which is set to one step by default, so the policy is updated every step. Assume we performed action from the state and thus reached state ′. As the goal of the algorithm is to minimise the energy of the structure (the single feature used), the reward for this step is defined as the energy drop from the current state to the new state ′: