class_docstr: PyTorch module, but with uncertainties.
  
  This base class provides useful tools to handle search spaces built on top of PyTorch modules,
  including methods like :meth:`simplify`, :meth:`freeze`.
  
  :class:`MutableModule` can have dangling mutables registered on it via :meth:`add_mutable`.
class_docstr_tok: ["PyTorch","module",",","but","with","uncertainties",".","This","base","class","provides","useful","tools","to","handle","search","spaces","built","on","top","of","PyTorch","modules",",","including","methods","like",":","meth",":","`","simplify","`",",",":","meth",":","`","freeze","`",".",":","class",":","`","MutableModule","`","can","have","dangling","mutables","registered","on","it","via",":","meth",":","`","add_mutable","`","."]
docstr: Register a mutable to this module.
  This is often used to add dangling variables that are not parameters of any :class:`ParametrizedModule`.
  
  If the mutable is also happens to be a submodule of type :class:`MutableModule`,
  it can be registered in the same way as PyTorch (i.e., ``self.xxx = mutable``). No need to add it again here.
  
  Examples
  --------
  In practice, this method is often used together with :func:`~nni.mutable.ensure_frozen`.
  
  >>> class MyModule(MutableModule):
  ...     def __init__(self):
  ...         super().__init__()
  ...         token_size = nni.choice('t', [4, 8, 16])        # Categorical variable here
  ...         self.add_mutable(token_size)                    # Register the mutable to this module.
  ...         real_token_size = ensure_frozen(token_size)     # Real number. 4 during dry run. 4, 8 or 16 during search.
  ...         self.token = nn.Parameter(torch.randn(real_token_size, 1))
  
  .. tip::
  
      Note that :func:`~nni.mutable.ensure_frozen` must be used under a :func:`~nni.mutable.frozen_context`.
      The easiest way to do so is to invoke it within initialization of a :class:`ModelSpace`.
  
  Warnings
  --------
  Arbitrary :meth:`add_mutable` is not supported for :class:`~nni.nas.space.GraphModelSpace`.
code: def add_mutable(self, mutable: T) -> T:
      """Register a mutable to this module.
      This is often used to add dangling variables that are not parameters of any :class:`ParametrizedModule`.
  
      If the mutable is also happens to be a submodule of type :class:`MutableModule`,
      it can be registered in the same way as PyTorch (i.e., ``self.xxx = mutable``). No need to add it again here.
  
      Examples
      --------
      In practice, this method is often used together with :func:`~nni.mutable.ensure_frozen`.
  
      >>> class MyModule(MutableModule):
      ...     def __init__(self):
      ...         super().__init__()
      ...         token_size = nni.choice('t', [4, 8, 16])        # Categorical variable here
      ...         self.add_mutable(token_size)                    # Register the mutable to this module.
      ...         real_token_size = ensure_frozen(token_size)     # Real number. 4 during dry run. 4, 8 or 16 during search.
      ...         self.token = nn.Parameter(torch.randn(real_token_size, 1))
  
      .. tip::
  
          Note that :func:`~nni.mutable.ensure_frozen` must be used under a :func:`~nni.mutable.frozen_context`.
          The easiest way to do so is to invoke it within initialization of a :class:`ModelSpace`.
  
      Warnings
      --------
      Arbitrary :meth:`add_mutable` is not supported for :class:`~nni.nas.space.GraphModelSpace`.
      """
  
      # NOTE:
      # This method can be potentially useful for every mutable. But I leave it here for the following reasons:
      # 1. If `_mutables` is to be put into base class, all subclasses need to call `super().__init__()`.
      # 2. Most of other mutables are "leaves". They do not have children. I don't agree that all mutables are "containers".
      # 3. To support nested mutables (like NestedCategorical), designs are needed (e.g., can constants be added via `add_mutable`?).
      # 4. The interface can be easily extended to base class if needed in future, without breaking backward compatibility.
  
      if isinstance(mutable, MutableModule):
          # If the mutable is also a module, it will be added automatically.
          return mutable
  
      if not isinstance(mutable, Mutable):
          raise TypeError(
              f"Expected Mutable, got {type(mutable)}: {mutable}"
          )
  
      # Dynamically create it here to avoid an __init__ call.
      # Also avoiding dependencies for __init__ in ParametrizedModule.
      if not hasattr(self, "_mutables"):
          self._mutables: List[Mutable] = []
  
      self._mutables.append(mutable)
  
      # Disable dry run when:
      # 1. Inside a model_context; the `current_model()` needs to be read-only.
      # 2. ensure_frozen_strict is disabled. The `default()` can be called when `ensure_frozen()` is called.
      if current_model() is None:
          # We will "dry run" the mutable here, to obtain a default value for it.
          context = frozen_context.current()
          if context is None:
              if frozen._ENSURE_FROZEN_STRICT:
                  _logger.warning(
                      "No context found when `add_mutable()`. "
                      "You are probably adding a MutableModule outside the `__init__` of a ModelSpace. "
                      "This can possibly make the MutableModule untrackable and inconsistent: %s",
                      mutable,
                  )
          else:
              context_before_keys = set(context.keys())
  
              try:
                  mutable.robust_default(context)
              except ValueError:
                  _logger.error(
                      "Error when trying to dry run the mutable: %r. It could be conflicted with current context: %s.",
                      mutable,
                      context,
                  )
                  raise
  
              frozen_context.update(
                  {
                      key: value
                      for key, value in context.items()
                      if key not in context_before_keys
                  }
              )
  
      return mutable
  
imports: import copy
  import itertools
  import inspect
  import logging
  from typing import Any, Callable, Type, Iterable, Dict, Tuple, List, Optional, TypeVar
  import torch
  from torch import nn
  from nni.mutable import Mutable, LabeledMutable, Sample, SampleValidationError, ensure_frozen, frozen_context, label_scope, frozen
  from nni.nas.space import BaseModelSpace, current_model, model_context
  from nni.common.serializer import SerializableObject, _formulate_arguments, _copy_class_wrapper_attributes
label_desc: Use image node_id 3 for calling the MutableModule obj's underlying member method code with example usage: obj.add_mutable(mutable) and returns: mutable, mutable