We investigated the reaction pathway finding problem, where, given one EQ node for each reactant and product, a kinetically feasible reaction pathway connecting these two nodes, starting from the reactant, was efficiently identified. Reinforcement learning algorithms based on various search policies were introduced, and their statistical performances were evaluated using a large-scale benchmark dataset. A score representing the energetic or structural similarity of each EQ node to the product's (goal's) EQ node was defined, and for each iteration step in the search, the average scores and standard deviations of node selection candidates were evaluated based on the information of the previously discovered reaction path network. According to these evaluation results, various policies were constructed to select the node for the next reaction path sampling. Exploration-exploitation balanced policies, including UCB variants possessing several parameters, and parameter-free methods, such as TS, PI, and EI, were introduced. As references, baseline strategies, such as Random, Greedy, and Uniform, were also investigated. The performance evaluations based on the goal finding rates as a function of iteration step showed that for the Passerini reaction dataset, Uniform and Random performed relatively well among the baselines, while Greedy performed poorly. In contrast, for the Strecker reaction dataset, Greedy performed well, while Uniform and, particularly, Random performed poorly. The baseline performances strongly depend on the target system, with the former and latter favoring exploration and exploitation, respectively. However, the exploration-exploitation balanced policies such as TS, PI, and EI demonstrated stable and high performance. Although UCB (C=1)'s performance was similar to that of exploitation term in UCB, a performance similar to that of TS could be achieved using a higher C parameter value in UCB. Additionally, the nodes selected First, a hierarchical search policy was investigated using molecular-bonding-pattern-based node clustering, revealing that this policy sometimes improved the search's performance.