The two Crystal Structure Prediction tools used in this paper, FUSE and MC-EMMA, are based on the basin-hopping method. Basin-hopping involves a Monte Carlo type search for low energy structures outlined in Fig. . The search starts by constructing an initial modular structure and performing a local geometry optimisation using a computational chemistry code, which takes the initial structure and locates the nearest local energy minimum. In this work, local optimisations are performed with GULP using interatomic potentials that we have developed and used previously . To conduct the search, the routine will take the locally optimised initial structure and modify it using a permitted action (see FUSE and MC-EMMA below for actions used in this work), e.g., to swap the positions of two atoms to create a new "trial structure" for comparison with the current structure. The energy of the trial structure is assessed using the same local optimisation. In the rare event when the local optimisation fails to complete, the trial structure is rejected, and a new trial structure is generated. To decide if the trial structure is accepted, we compare the arising energy of the locally optimised trial structure with that of the current structure. If the trial structure energy is lower, then the trial structure is always accepted as the new current structure. If the trial structure energy is higher, then there is a chance that the search will accept an increase in energy, governed by the condition "Rand" > ( / ) , where "Rand" is a randomly generated number within the range of 0 to 1, ΔE is the energy difference between the trial and current structures and kT is the "temperature" parameter of the search. Higher temperature parameters will result in an increased probability that the search will accept an increase in energy. In this work kT is 0.02eV for FUSE and 0.025eV for MC-EMMA as used in and respectively. This routine is then repeated until either a user-defined number of structures has been generated, or a break condition has been met, e.g., a set number of structures have been generated since the current global minimum was located. both the FUSE and MC-EMMA structure prediction codes described below. "Rand" is a randomly generated number between 0 and 1, ΔE indicates the difference in energy between the trial structure and the current structure, kT is the "temperature" parameter for the system (set by the user). The "Break" condition is triggered when the maximum allowable number of consecutive rejected structures is reached (defined by user). The text in red shows how RL-CSP is embedded into BH; the RL-CSP policy is used instead of the fixed probability distribution to select the action to alter the structure that maximises the reward and then the policy is updated based on the results of the energy calculation.