For our choice of state space, we use the current structure as the state and its energy as a feature, since it is intuitively an important factor in deciding what actions are likely to be good for the current structure and is computed as part of the BH routines and therefore does not add an unnecessary overhead. Energy is normalised (see Methods) to ensure that RL-CSP will work with the same set of hyperparameters across a range of compositions, as the range of possible energies changes between compositions.