MC-EMMA uses a BH routine that is similar to FUSE, however it was designed for larger crystal structures with up to 160 atoms while FUSE was tested with the structures up to 50-80 atoms depending on the composition. Although fewer steps are usually required to find the global minimum (~1,500-2,000 steps for fixed policies) than those in most FUSE compositions, the relaxation of the larger structures is more time-consuming and so MC-EMMA presents a different challenge for RL, which is to improve the starting policy using less data. As in the evaluation of RL for FUSE, we compare three versions of policy in MC-EMMA: the original hand-tuned fixed policy (Original), the uniform policy (Uniform) and the RL-CSP policy that starts learning from the Uniform. The action probability distributions for the Original policy are shown in Supplementary Fig. . We test MC-EMMA policies at the composition YBa2Ca2Fe5O13, which the Original policy was tuned for. As in the FUSE benchmarking, we use 40 runs of MC-EMMA with each policy. However, in FUSE we observed that RL-CSP does not have enough time to learn the policy at compositions where Nmean is less than ~10,000 steps for the Uniform policy (Sr4Ti3O10 and Y2TiO5). Taking this into account, we boost the learning process in RL-CSP by increasing the learning rate (see Methods) in comparison to the FUSE settings and sharing the policy learning process across 40 runs. All 40 runs have different structure sequences and perform BH steps in parallel independently but use the same RL-CSP policy and update it one by one (the details of the policy sharing are provided in the Methods section). This helps accelerate the learning process; however, it may lead to a poorer performing policy as FUSE testing shows that the dynamic policy varies at different steps as the run proceeds and does not necessarily converge (see Fig. ).