The reward function for the final state is then calculated (eq 2) and the score back propagated through the tree, updating the Q and N values of all parent states between the final state and the root state (target compound). N is the total number of compounds in the state, Nin_stock Is the number of compounds that are in stock.