Initialize search episodes from promising candidates. To bias the initial state distribution ρ 0 to favor molecules that can derive more novel high-quality molecules, we keep an upper-confidencebound (UCB) score for each initial molecule in the frontier F . We record the number of times we initiate a search N (x, t) from a molecule x ∈ F , and the number of molecules qualified for adding to C that is found in an episode strating from x: R(x, t). Here t = x∈ρ0 N (x) is the total number of search episodes. The UCB score of the initial molecule m is calculated by: