Once the action a is selected, the search policy terminates and the local reward r is computed according to Eq. ( ) and the weights w and v are updated according to Eq. ( ) and Eq. ( ), respectively. Upon taking an action a = (p, q), S 1 can be updated by removing p and adding q, and the next candidate external determinant in S 2 can be considered. This prevents the need to re-construct S 1 and S 2 at each step in an episode. The episode terminates when no further suitable candidate actions can be found, or once the space is exhausted.