The final part of the RL approach is the on-the-fly update of the policy parameters during the CSP run. RL-CSP starts from the uniform policy which allocates an equal probability to every action and does not depend on the structure. Then the algorithm updates the policy parameters via a gradient ascent step that changes the policy so that, given the structure with its energy, the actions that yield positive rewards get an increased probability in the states with this energy. Standard RL methods, such as entropy regularization and an -greedy policy , are utilized as well to guarantee policy exploration, which is a critical aspect of RL . Further details are provided in the Methods section.