alpaca/alpacka/agents/stochastic_mcts.py
def _run_pass(self, root, observation):
        """Runs a pass of MCTS.

        A pass consists of:
            1. Tree traversal to find a leaf.
            2. Expansion of the leaf, adding its successor states to the tree
               and rating them.
            3. Backpropagation of the value of the best child of the old leaf.

        During leaf expansion, new children are rated only using
        the rate_new_leaves_fn - no actual stepping into those states in the
        environment takes place for efficiency, so that rate_new_leaves_fn can
        be implemented by running a neural network that rates all children of
        a given node at the same time. This means that those new leaves cannot
        be matched with GraphNodes yet, which leads to the special case 2.
        below.

        Special cases:
            1. In case of a "done", traversal is interrupted, the leaf is not
               expanded and value 0 is backpropagated.
            2. When graph_mode is turned on and the expanded leaf turns out to
               be an already visited node in the graph, we don't backpropagate
               value (because it's unclear which one to backpropagate), so the
               next pass can follow the same path, go through this node and find
               a "true" leaf. Rewards are backpropagated however, because
               they're still valid.
            3. When avoid_loops is turned on and during tree traversal no action
               can be chosen without making a loop, value defined in the
               loop_penalty  option is backpropagated from the node.

        Args:
            root (TreeNode): Root node.
            observation (np.ndarray): Observation collected at the root.

        Yields:
            Network prediction requests.
        """
        path = []
        try:
            (observation, done, visited) = self._traverse(
                root, observation, path
            )
            (_, leaf) = path[-1]
            quality = yield from self._expand_leaf(
                leaf, observation, done, visited
            )
        except DeadEnd:
            quality = self._loop_penalty
        self._backpropagate(quality, path)
        # Go back to the root state.
        self._model.restore_state(self._root_state)
Runs a pass of MCTS.

        A pass consists of:
            1. Tree traversal to find a leaf.
            2. Expansion of the leaf, adding its successor states to the tree
               and rating them.
            3. Backpropagation of the value of the best child of the old leaf.

        During leaf expansion, new children are rated only using
        the rate_new_leaves_fn - no actual stepping into those states in the
        environment takes place for efficiency, so that rate_new_leaves_fn can
        be implemented by running a neural network that rates all children of
        a given node at the same time. This means that those new leaves cannot
        be matched with GraphNodes yet, which leads to the special case 2.
        below.

        Special cases:
            1. In case of a "done", traversal is interrupted, the leaf is not
               expanded and value 0 is backpropagated.
            2. When graph_mode is turned on and the expanded leaf turns out to
               be an already visited node in the graph, we don't backpropagate
               value (because it's unclear which one to backpropagate), so the
               next pass can follow the same path, go through this node and find
               a "true" leaf. Rewards are backpropagated however, because
               they're still valid.
            3. When avoid_loops is turned on and during tree traversal no action
               can be chosen without making a loop, value defined in the
               loop_penalty  option is backpropagated from the node.

        Args:
            root (TreeNode): Root node.
            observation (np.ndarray): Observation collected at the root.

        Yields:
            Network prediction requests.
        
Runs a pass of MCTS.
A pass consists of:
1. Tree traversal to find a leaf.
2. Expansion of the leaf, adding its successor states to the tree
and rating them.
3. Backpropagation of the value of the best child of the old leaf.

During leaf expansion, new children are rated only using
the rate_new_leaves_fn - no actual stepping into those states in the
environment takes place for efficiency, so that rate_new_leaves_fn can
be implemented by running a neural network that rates all children of
a given node at the same time. This means that those new leaves cannot
be matched with GraphNodes yet, which leads to the special case 2.
below.

Special cases:
1. In case of a "done", traversal is interrupted, the leaf is not
expanded and value 0 is backpropagated.
2. When graph_mode is turned on and the expanded leaf turns out to
be an already visited node in the graph, we don't backpropagate
value (because it's unclear which one to backpropagate), so the
next pass can follow the same path, go through this node and find
a "true" leaf. Rewards are backpropagated however, because
they're still valid.
3. When avoid_loops is turned on and during tree traversal no action
can be chosen without making a loop, value defined in the
loop_penalty  option is backpropagated from the node.
Runs a pass of MCTS . A pass consists of : 1 . Tree traversal to find a leaf . 2 . Expansion of the leaf adding its successor states to the tree and rating them . 3 . Backpropagation of the value of the best child of the old leaf . During leaf expansion new children are rated only using the rate_new_leaves_fn - no actual stepping into those states in the environment takes place for efficiency so that rate_new_leaves_fn can be implemented by running a neural network that rates all children of a given node at the same time . This means that those new leaves cannot be matched with GraphNodes yet which leads to the special case 2 . below . Special cases : 1 . In case of a " done " traversal is interrupted the leaf is not expanded and value 0 is backpropagated . 2 . When graph_mode is turned on and the expanded leaf turns out to be an already visited node in the graph we don ' t backpropagate value ( because it ' s unclear which one to backpropagate ) so the next pass can follow the same path go through this node and find a " true " leaf . Rewards are backpropagated however because they ' re still valid . 3 . When avoid_loops is turned on and during tree traversal no action can be chosen without making a loop value defined in the loop_penalty option is backpropagated from the node .
def _run_pass(self, root, observation):
        path = []
        try:
            (observation, done, visited) = self._traverse(
                root, observation, path
            )
            (_, leaf) = path[-1]
            quality = yield from self._expand_leaf(
                leaf, observation, done, visited
            )
        except DeadEnd:
            quality = self._loop_penalty
        self._backpropagate(quality, path)
        self._model.restore_state(self._root_state)
def _run_pass ( self , root , observation ) : path = [ ] try : ( observation , done , visited ) = self . _traverse ( root , observation , path ) ( _ , leaf ) = path [ - 1 ] quality = yield from self . _expand_leaf ( leaf , observation , done , visited ) except DeadEnd : quality = self . _loop_penalty self . _backpropagate ( quality , path ) self . _model . restore_state ( self . _root_state )
Runs a pass of MCTS.
Runs a pass of MCTS .
"""Runs a pass of MCTS.

        A pass consists of:
            1. Tree traversal to find a leaf.
            2. Expansion of the leaf, adding its successor states to the tree
               and rating them.
            3. Backpropagation of the value of the best child of the old leaf.

        During leaf expansion, new children are rated only using
        the rate_new_leaves_fn - no actual stepping into those states in the
        environment takes place for efficiency, so that rate_new_leaves_fn can
        be implemented by running a neural network that rates all children of
        a given node at the same time. This means that those new leaves cannot
        be matched with GraphNodes yet, which leads to the special case 2.
        below.

        Special cases:
            1. In case of a "done", traversal is interrupted, the leaf is not
               expanded and value 0 is backpropagated.
            2. When graph_mode is turned on and the expanded leaf turns out to
               be an already visited node in the graph, we don't backpropagate
               value (because it's unclear which one to backpropagate), so the
               next pass can follow the same path, go through this node and find
               a "true" leaf. Rewards are backpropagated however, because
               they're still valid.
            3. When avoid_loops is turned on and during tree traversal no action
               can be chosen without making a loop, value defined in the
               loop_penalty  option is backpropagated from the node.

        Args:
            root (TreeNode): Root node.
            observation (np.ndarray): Observation collected at the root.

        Yields:
            Network prediction requests.
        """ # Go back to the root state.
Network prediction requests.
Network prediction requests .
self
root
None
observation
Observation collected at the root.
Observation collected at the root .