rl.mcts.mcts_node.MCTSNode Class Reference

Node class for Monte Carlo Tree Search. More...

Public Member Functions
	__init__ (self, state, parent=None, action=None, depth=0)
	Initialize a new MCTS node.
	is_root (self)
	Check if this node is the root of the tree.
	is_terminal (self)
	Check if this node represents a terminal state.
	is_fully_expanded (self)
	Check if all possible actions have been tried from this node.
	get_untried_action (self)
	Get list of actions that haven't been tried yet from this node.
	expand (self, tuple candidate)
	Expand the node by adding a new child for the given action.
	add_child (self, child)
	Add a child node to this node.
	best_child (self, exploration_weight=1.0)
	Select the best child node using UCT (Upper Confidence bounds applied to Trees)
	backpropagate (self, reward)
	Backpropagate the reward up the tree to all ancestors.

Public Attributes
	state = state
	parent = parent
	action = action
	depth = depth
list	children = []
int	visits = 0
float	total_reward = 0.0
int	children = 0:

Detailed Description

Node class for Monte Carlo Tree Search.

Each node represents a state in the search tree and maintains information about visits, rewards, and possible actions.

Constructor & Destructor Documentation

__init__()

rl.mcts.mcts_node.MCTSNode.__init__	(		self,
			state,
			parent = None,
			action = None,
			depth = 0 )

Initialize a new MCTS node.

Parameters

state	Current problem state
parent	Parent node in the search tree
action	Action taken to reach this node
depth	Depth of this node in the tree

    def __init__(self, state, parent=None, action=None, depth=0):
        """!
        @brief Initialize a new MCTS node
        @param state Current problem state
        @param parent Parent node in the search tree
        @param action Action taken to reach this node
        @param depth Depth of this node in the tree
        """
        self.state = state # Current state (later observation)
        self.parent = parent # Parent node
        self.action = action # Action taken to reach this node
        self.depth = depth # Depth in the tree
        self.children = [] # List of child nodes
        self.visits = 0 # Number of visits to this node
        self.total_reward = 0.0 # Total reward accumulated from this node
 

Member Function Documentation

add_child()

rl.mcts.mcts_node.MCTSNode.add_child	(		self,
			child )

Add a child node to this node.

Parameters

child

Child node to add

    def add_child(self, child):
        """!
        @brief Add a child node to this node
        @param child Child node to add
        """
        self.children.append(child)
 

backpropagate()

rl.mcts.mcts_node.MCTSNode.backpropagate	(		self,
			reward )

Backpropagate the reward up the tree to all ancestors.

Parameters

reward

Reward value to propagate

    def backpropagate(self, reward):
        """!
        @brief Backpropagate the reward up the tree to all ancestors
        @param reward Reward value to propagate
        """
        self.visits += 1
        self.total_reward += reward
        if self.parent:
            self.parent.backpropagate(reward)

best_child()

rl.mcts.mcts_node.MCTSNode.best_child	(		self,
			exploration_weight = 1.0 )

Select the best child node using UCT (Upper Confidence bounds applied to Trees)

Parameters

exploration_weight

Weight for exploration vs exploitation trade-off

Returns

Best child node according to UCT formula

    def best_child(self, exploration_weight=1.0):
        """!
        @brief Select the best child node using UCT (Upper Confidence bounds applied to Trees)
        @param exploration_weight Weight for exploration vs exploitation trade-off
        @return Best child node according to UCT formula
        """
        # Example: UCT formula
        best_score = float('-inf')
        best = None
        for child in self.children:
            if child.visits == 0:
                score = float('inf')
            else:
                exploitation = child.total_reward / child.visits
                exploration = exploration_weight * math.sqrt(math.log(self.visits) / child.visits)
                score = exploitation + exploration
            if score > best_score:
                best_score = score
                best = child
        return best
 

expand()

rl.mcts.mcts_node.MCTSNode.expand	(		self,
		tuple	candidate )

Expand the node by adding a new child for the given action.

Parameters

candidate

Tuple of (action_name, parameters) to expand

Returns

The newly created child node

    def expand(self, candidate: tuple):
        """!
        @brief Expand the node by adding a new child for the given action
        @param candidate Tuple of (action_name, parameters) to expand
        @return The newly created child node
        """
        new_state = self.state.copy()  # Create copy
        new_state.apply_action(candidate[0], candidate[1])  # Apply action on copy
        child_node = MCTSNode(state=new_state, parent=self, action=candidate, depth=self.depth + 1)
        self.add_child(child_node)
        return child_node
 

get_untried_action()

rl.mcts.mcts_node.MCTSNode.get_untried_action

(

self

)

Get list of actions that haven't been tried yet from this node.

Returns

List of untried (action_name, parameters) tuples

    def get_untried_action(self):
        """!
        @brief Get list of actions that haven't been tried yet from this node
        @return List of untried (action_name, parameters) tuples
        """
        # If root node has a specific action without parameters
        if self.is_root() and self.action is not None and self.action[1] is None:
            # Only return parameters for this specific action
            action_name = self.action[0]
            all_params = self.state.enumerate_valid_params(action_name)
            tried_params = [child.action[1] for child in self.children]
            # Return only untried parameters
            return [(action_name, param) for param in all_params if param not in tried_params]
        else:
            # Normal behavior for other nodes
            all_possible_actions = self.state.get_possible_actions()
            tried_actions = [child.action for child in self.children]
            return [action for action in all_possible_actions if action not in tried_actions]
 

is_fully_expanded()

rl.mcts.mcts_node.MCTSNode.is_fully_expanded

(

self

)

Check if all possible actions have been tried from this node.

Returns

True if fully expanded, False otherwise

    def is_fully_expanded(self):
        """!
        @brief Check if all possible actions have been tried from this node
        @return True if fully expanded, False otherwise
        """
        return len(self.get_untried_action()) == 0
        

is_root()

rl.mcts.mcts_node.MCTSNode.is_root

(

self

)

Check if this node is the root of the tree.

Returns

True if this is the root node, False otherwise

    def is_root(self):
        """!
        @brief Check if this node is the root of the tree
        @return True if this is the root node, False otherwise
        """
        return self.parent is None
 

is_terminal()

rl.mcts.mcts_node.MCTSNode.is_terminal

(

self

)

Check if this node represents a terminal state.

Returns

True if this is a terminal state, False otherwise

    def is_terminal(self):
        """!
        @brief Check if this node represents a terminal state
        @return True if this is a terminal state, False otherwise
        """
        return self.state.is_terminal()
 

Member Data Documentation

action

rl.mcts.mcts_node.MCTSNode.action = action

children [1/2]

list rl.mcts.mcts_node.MCTSNode.children = []

children [2/2]

int rl.mcts.mcts_node.MCTSNode.children = 0:

depth

rl.mcts.mcts_node.MCTSNode.depth = depth

parent

rl.mcts.mcts_node.MCTSNode.parent = parent

state

rl.mcts.mcts_node.MCTSNode.state = state

total_reward

float rl.mcts.mcts_node.MCTSNode.total_reward = 0.0

visits

int rl.mcts.mcts_node.MCTSNode.visits = 0

---

The documentation for this class was generated from the following file:

• rl/mcts/mcts_node.py