docstr: This trainer implements the semi-supervised task training method of GraphCL for graph-level tasks with pretraining and finetuning.
  GraphCL is a contrastive method proposed in the paper `Graph Contrastive Learning with Augmentations` in nips 2020.
  <https://proceedings.neurips.cc/paper/2020/hash/3fe230348e9a12c13120749e3f9fa4cd-Abstract.html>
  
  Parameters
  ----------
  model:
      Models can be `str`, `autogl.module.model.encoders.BaseEncoderMaintainer` or a tuple of (encoder, decoder) 
      if need to specify both encoder and decoder. Encoder can be `str` or
      `autogl.module.model.encoders.BaseEncoderMaintainer`, and decoder can be `str`
      or `autogl.module.model.decoders.BaseDecoderMaintainer`.
      If only encoder is specified, decoder will be default to "sumpoolmlp"
  prediction_model: `BaseAutoModel`, `str` or None
      A model used to finetuning
      Only required if `node_level` = True.
  num_features: `int` or None
      The number of features in dataset.
  num_classes: `int` or None
      The number of classes in dataset. Only required when doing semi-supervised tasks.
  num_graph_features: `int` or None
      The number of graph level features in dataset.
  device: `torch.device` or `str`
      The device this trainer will use.
      When `device` = "auto", if GPU exists in the device and dependency is installed, 
      the trainer will give priority to GPU, otherwise CPU will be used
  feval: a sequence of `str` or a sequence of `Evaluation`
      The evaluation methods.
  loss: `str` or `Callable`
      The loss function or the learning objective of contrastive model.
  f_loss: `str` or `Callable`
      The loss function during finetuning stage.
  views_fn: a list of `str`, a list of `Callable` or None
      List of functions or augmentation methods to generate views from give graphs.
      In GraphCL, the number of views_fn should be 2.
      If the element of `views_fn` is str, the options should be "dropN", "permE", "subgraph", "maskN", "random2", "random3", "random4" or None.
  aug_ratio: a list of `float` or a `float`
      The ratio of augmentations, a number between [0, 1).
      If aug_ratio is set as a float, the value will shared by all views_fn.
      If aug_ratio is set as a list of float, the value of this list and views_fn one to one correspondence.
  z_dim: `int`
      The dimension of graph-level representations
  tau: `int`
      The temperature parameter in NT_Xent loss. Only used when `loss` = "NT_Xent"
  model_path: `str` or None
      The directory to restore the saved model.
      If `model_path` = None, the model will not be saved.
  num_workers: `int`
      Number of workers.
  batch_size: `int`
      Batch size for pretraining and inference.
  p_optim: `str` or `torch.optim.Optimizer`
      Optimizer for pretraining.
  p_lr: `float`
      Pretraining learning rate.
  p_lr_scheduler_type: `str`
      Scheduler type for pretraining learning rate.
  p_weight_decay: `float`
      Pretraining weight decay rate.
  p_epoch: `int`
      Pretraining epochs number.
  p_early_stopping: `int`
      Pretraining early stopping round.
  f_optim: `str` or `torch.optim.Optimizer`
      Optimizer for finetuning.
  f_lr: `float`
      Finetuning learning rate.
  f_lr_scheduler_type: `str`
      Scheduler type for finetuning learning rate.
  f_weight_decay: `float`
      Finetuning weight decay rate.
  f_epoch: `int`
      Finetuning epochs number.
  f_early_stopping: `int`
      Finetuning early stopping round.
  init: `bool`
      Whether to initialize the model.
docstr_tok: ["This","trainer","implements","the","semi-supervised","task","training","method","of","GraphCL","for","graph-level","tasks","with","pretraining","and","finetuning",".","GraphCL","is","a","contrastive","method","proposed","in","the","paper","`","Graph","Contrastive","Learning","with","Augmentations","`","in","nips","2020",".","<","https",":","\/\/proceedings.neurips.cc\/paper\/2020\/hash\/3fe230348e9a12c13120749e3f9fa4cd-Abstract.html",">","Parameters","--","--","--","--","--","model",":","Models","can","be","`","str","`",",","`","autogl.module.model.encoders.BaseEncoderMaintainer","`","or","a","tuple","of","(","encoder",",","decoder",")","if","need","to","specify","both","encoder","and","decoder",".","Encoder","can","be","`","str","`","or","`","autogl.module.model.encoders.BaseEncoderMaintainer","`",",","and","decoder","can","be","`","str","`","or","`","autogl.module.model.decoders.BaseDecoderMaintainer","`",".","If","only","encoder","is","specified",",","decoder","will","be","default","to","``","sumpoolmlp","''","prediction_model",":","`","BaseAutoModel","`",",","`","str","`","or","None","A","model","used","to","finetuning","Only","required","if","`","node_level","`","=","True",".","num_features",":","`","int","`","or","None","The","number","of","features","in","dataset",".","num_classes",":","`","int","`","or","None","The","number","of","classes","in","dataset",".","Only","required","when","doing","semi-supervised","tasks",".","num_graph_features",":","`","int","`","or","None","The","number","of","graph","level","features","in","dataset",".","device",":","`","torch.device","`","or","`","str","`","The","device","this","trainer","will","use",".","When","`","device","`","=","``","auto","''",",","if","GPU","exists","in","the","device","and","dependency","is","installed",",","the","trainer","will","give","priority","to","GPU",",","otherwise","CPU","will","be","used","feval",":","a","sequence","of","`","str","`","or","a","sequence","of","`","Evaluation","`","The","evaluation","methods",".","loss",":","`","str","`","or","`","Callable","`","The","loss","function","or","the","learning","objective","of","contrastive","model",".","f_loss",":","`","str","`","or","`","Callable","`","The","loss","function","during","finetuning","stage",".","views_fn",":","a","list","of","`","str","`",",","a","list","of","`","Callable","`","or","None","List","of","functions","or","augmentation","methods","to","generate","views","from","give","graphs",".","In","GraphCL",",","the","number","of","views_fn","should","be","2",".","If","the","element","of","`","views_fn","`","is","str",",","the","options","should","be","``","dropN","''",",","``","permE","''",",","``","subgraph","''",",","``","maskN","''",",","``","random2","''",",","``","random3","''",",","``","random4","''","or","None",".","aug_ratio",":","a","list","of","`","float","`","or","a","`","float","`","The","ratio","of","augmentations",",","a","number","between","[","0",",","1",")",".","If","aug_ratio","is","set","as","a","float",",","the","value","will","shared","by","all","views_fn",".","If","aug_ratio","is","set","as","a","list","of","float",",","the","value","of","this","list","and","views_fn","one","to","one","correspondence",".","z_dim",":","`","int","`","The","dimension","of","graph-level","representations","tau",":","`","int","`","The","temperature","parameter","in","NT_Xent","loss",".","Only","used","when","`","loss","`","=","``","NT_Xent","''","model_path",":","`","str","`","or","None","The","directory","to","restore","the","saved","model",".","If","`","model_path","`","=","None",",","the","model","will","not","be","saved",".","num_workers",":","`","int","`","Number","of","workers",".","batch_size",":","`","int","`","Batch","size","for","pretraining","and","inference",".","p_optim",":","`","str","`","or","`","torch.optim.Optimizer","`","Optimizer","for","pretraining",".","p_lr",":","`","float","`","Pretraining","learning","rate",".","p_lr_scheduler_type",":","`","str","`","Scheduler","type","for","pretraining","learning","rate",".","p_weight_decay",":","`","float","`","Pretraining","weight","decay","rate",".","p_epoch",":","`","int","`","Pretraining","epochs","number",".","p_early_stopping",":","`","int","`","Pretraining","early","stopping","round",".","f_optim",":","`","str","`","or","`","torch.optim.Optimizer","`","Optimizer","for","finetuning",".","f_lr",":","`","float","`","Finetuning","learning","rate",".","f_lr_scheduler_type",":","`","str","`","Scheduler","type","for","finetuning","learning","rate",".","f_weight_decay",":","`","float","`","Finetuning","weight","decay","rate",".","f_epoch",":","`","int","`","Finetuning","epochs","number",".","f_early_stopping",":","`","int","`","Finetuning","early","stopping","round",".","init",":","`","bool","`","Whether","to","initialize","the","model","."]
code: def __init__(
      self,
      model: Tuple[BaseEncoderMaintainer, BaseDecoderMaintainer] = None,
      prediction_head: Union[BaseDecoderMaintainer, str, None] = None,
      num_features: Union[int, None] = None,
      num_classes: Union[int, None] = None,
      num_graph_features: Union[int, None] = 0,
      device: Union[torch.device, str] = "auto",
      feval: Union[Sequence[str], Sequence[Type[Evaluation]]] = (Acc,),
      views_fn: Union[Sequence[str], Sequence[Callable], None] = None,
      aug_ratio: Union[float, Sequence[float]] = 0.2,
      z_dim: Union[int, None] = 128,
      num_workers: int = 0,
      batch_size: int = 128,
      eval_interval: int = 10,
      init: bool = False,
      *args,
      **kwargs,
  ):
      """
      This trainer implements the semi-supervised task training method of GraphCL for graph-level tasks with pretraining and finetuning.
      GraphCL is a contrastive method proposed in the paper `Graph Contrastive Learning with Augmentations` in nips 2020.
      <https://proceedings.neurips.cc/paper/2020/hash/3fe230348e9a12c13120749e3f9fa4cd-Abstract.html>
  
      Parameters
      ----------
      model:
          Models can be `str`, `autogl.module.model.encoders.BaseEncoderMaintainer` or a tuple of (encoder, decoder)
          if need to specify both encoder and decoder. Encoder can be `str` or
          `autogl.module.model.encoders.BaseEncoderMaintainer`, and decoder can be `str`
          or `autogl.module.model.decoders.BaseDecoderMaintainer`.
          If only encoder is specified, decoder will be default to "sumpoolmlp"
      prediction_model: `BaseAutoModel`, `str` or None
          A model used to finetuning
          Only required if `node_level` = True.
      num_features: `int` or None
          The number of features in dataset.
      num_classes: `int` or None
          The number of classes in dataset. Only required when doing semi-supervised tasks.
      num_graph_features: `int` or None
          The number of graph level features in dataset.
      device: `torch.device` or `str`
          The device this trainer will use.
          When `device` = "auto", if GPU exists in the device and dependency is installed,
          the trainer will give priority to GPU, otherwise CPU will be used
      feval: a sequence of `str` or a sequence of `Evaluation`
          The evaluation methods.
      loss: `str` or `Callable`
          The loss function or the learning objective of contrastive model.
      f_loss: `str` or `Callable`
          The loss function during finetuning stage.
      views_fn: a list of `str`, a list of `Callable` or None
          List of functions or augmentation methods to generate views from give graphs.
          In GraphCL, the number of views_fn should be 2.
          If the element of `views_fn` is str, the options should be "dropN", "permE", "subgraph", "maskN", "random2", "random3", "random4" or None.
      aug_ratio: a list of `float` or a `float`
          The ratio of augmentations, a number between [0, 1).
          If aug_ratio is set as a float, the value will shared by all views_fn.
          If aug_ratio is set as a list of float, the value of this list and views_fn one to one correspondence.
      z_dim: `int`
          The dimension of graph-level representations
      tau: `int`
          The temperature parameter in NT_Xent loss. Only used when `loss` = "NT_Xent"
      model_path: `str` or None
          The directory to restore the saved model.
          If `model_path` = None, the model will not be saved.
      num_workers: `int`
          Number of workers.
      batch_size: `int`
          Batch size for pretraining and inference.
      p_optim: `str` or `torch.optim.Optimizer`
          Optimizer for pretraining.
      p_lr: `float`
          Pretraining learning rate.
      p_lr_scheduler_type: `str`
          Scheduler type for pretraining learning rate.
      p_weight_decay: `float`
          Pretraining weight decay rate.
      p_epoch: `int`
          Pretraining epochs number.
      p_early_stopping: `int`
          Pretraining early stopping round.
      f_optim: `str` or `torch.optim.Optimizer`
          Optimizer for finetuning.
      f_lr: `float`
          Finetuning learning rate.
      f_lr_scheduler_type: `str`
          Scheduler type for finetuning learning rate.
      f_weight_decay: `float`
          Finetuning weight decay rate.
      f_epoch: `int`
          Finetuning epochs number.
      f_early_stopping: `int`
          Finetuning early stopping round.
      init: `bool`
          Whether to initialize the model.
      """
      # set encoder and decoder
      if isinstance(model, Tuple):
          encoder, decoder = model
      elif isinstance(model, BaseAutoModel):
          raise ValueError(
              "The GraphCL trainer must need an encoder and a decoder, so `model` shouldn't be an instance of `BaseAutoModel`"
          )
      else:
          encoder, decoder = model, "sumpoolmlp"
      self.eval_interval = eval_interval
      # init contrastive learning
      super().__init__(
          encoder=encoder,
          decoder=decoder,
          decoder_node=None,
          num_features=num_features,
          num_graph_features=num_graph_features,
          views_fn=views_fn,
          graph_level=True,
          node_level=False,
          device=device,
          feval=feval,
          z_dim=z_dim,
          z_node_dim=None,
          *args,
          **kwargs,
      )
      self.views_fn = views_fn
      self.aug_ratio = aug_ratio
      self._prediction_head = None
      self.num_classes = num_classes
      self.prediction_head = prediction_head
      # recording the information about dataloader
      self.batch_size = batch_size
      self.num_workers = num_workers
      if self.num_workers > 0:
          mp.set_start_method("fork", force=True)
  
      # hyper parameter space
      self.hyper_parameter_space = [
          {
              "parameterName": "batch_size",
              "type": "INTEGER",
              "maxValue": 128,
              "minValue": 32,
              "scalingType": "LOG",
          },
          {
              "parameterName": "p_epoch",
              "type": "INTEGER",
              "maxValue": 150,
              "minValue": 50,
              "scalingType": "LINEAR",
          },
          {
              "parameterName": "p_early_stopping_round",
              "type": "INTEGER",
              "maxValue": 30,
              "minValue": 10,
              "scalingType": "LINEAR",
          },
          {
              "parameterName": "p_lr",
              "type": "DOUBLE",
              "maxValue": 1e-2,
              "minValue": 1e-4,
              "scalingType": "LOG",
          },
          {
              "parameterName": "p_weight_decay",
              "type": "DOUBLE",
              "maxValue": 5e-3,
              "minValue": 5e-8,
              "scalingType": "LOG",
          },
          {
              "parameterName": "f_epoch",
              "type": "INTEGER",
              "maxValue": 150,
              "minValue": 50,
              "scalingType": "LINEAR",
          },
          {
              "parameterName": "f_early_stopping_round",
              "type": "INTEGER",
              "maxValue": 100,
              "minValue": 10,
              "scalingType": "LINEAR",
          },
          {
              "parameterName": "f_lr",
              "type": "DOUBLE",
              "maxValue": 1e-2,
              "minValue": 1e-4,
              "scalingType": "LOG",
          },
          {
              "parameterName": "f_weight_decay",
              "type": "DOUBLE",
              "maxValue": 5e-3,
              "minValue": 0,
              "scalingType": "LOG",
          },
      ]
      self.hyper_parameters = {
          "batch_size": self.batch_size,
          "p_epoch": self.p_epoch,
          "p_early_stopping_round": self.p_early_stopping_round,
          "p_lr": self.p_lr,
          "p_weight_decay": self.p_weight_decay,
          "f_epoch": self.f_epoch,
          "f_early_stopping_round": self.f_early_stopping_round,
          "f_lr": self.f_lr,
          "f_weight_decay": self.f_weight_decay,
      }
      self.args = args
      self.kwargs = kwargs
      if init:
          self.initialize()
  
imports: import os
  import torch
  import logging
  import torch.nn.functional
  import torch.utils.data
  import numpy
  import typing
  import torch.multiprocessing
  from typing import Union, Tuple, Sequence, Type, Callable
  from tqdm import trange
  from copy import deepcopy
  from .base import BaseContrastiveTrainer
  from ..None import register_trainer
  from ..base import EarlyStopping, _DummyModel
  from ..evaluation import Evaluation, get_feval, Acc
  from ...model import BaseAutoModel, BaseEncoderMaintainer, BaseDecoderMaintainer, DecoderUniversalRegistry
  from ....datasets import utils
  from ....utils import get_logger
  from torch_geometric.nn.glob import global_add_pool, global_max_pool, global_mean_pool
label_desc: Use image node_id 1 to create a new GraphCLUnsupervisedTrainer object from inherited base classes: BaseContrastiveTrainer with example: obj = GraphCLUnsupervisedTrainer(model, prediction_head, num_features, num_classes, num_graph_features, device, feval, views_fn, aug_ratio, z_dim, num_workers, batch_size, eval_interval, init)