class_docstr: PatchTST
  
  The PatchTST model is an efficient Transformer-based model for multivariate time series forecasting.
  
  It is based on two key components:
  - segmentation of time series into windows (patches) which are served as input tokens to Transformer
  - channel-independence, where each channel contains a single univariate time series.
  
  **Parameters:**<br>
  `h`: int, Forecast horizon. <br>
  `input_size`: int, autorregresive inputs size, y=[1,2,3,4] input_size=2 -> y_[t-2:t]=[1,2].<br>
  `stat_exog_list`: str list, static exogenous columns.<br>
  `hist_exog_list`: str list, historic exogenous columns.<br>
  `futr_exog_list`: str list, future exogenous columns.<br>
  `exclude_insample_y`: bool=False, the model skips the autoregressive features y[t-input_size:t] if True.<br>
  `encoder_layers`: int, number of layers for encoder.<br>
  `n_heads`: int=16, number of multi-head's attention.<br>
  `hidden_size`: int=128, units of embeddings and encoders.<br>
  `linear_hidden_size`: int=256, units of linear layer.<br>
  `dropout`: float=0.1, dropout rate for residual connection.<br>
  `fc_dropout`: float=0.1, dropout rate for linear layer.<br>
  `head_dropout`: float=0.1, dropout rate for Flatten head layer.<br>
  `attn_dropout`: float=0.1, dropout rate for attention layer.<br>
  `patch_len`: int=32, length of patch.<br>
  `stride`: int=16, stride of patch.<br>
  `revin`: bool=True, bool to use RevIn.<br>
  `revin_affine`: bool=False, bool to use affine in RevIn.<br>
  `revin_substract_last`: bool=False, bool to use substract last in RevIn.<br>
  `activation`: str='ReLU', activation from ['gelu','relu'].<br>
  `res_attention`: bool=False, bool to use residual attention.<br>
  `batch_normalization`: bool=False, bool to use batch normalization.<br>
  `learn_pos_embedding`: bool=True, bool to learn positional embedding.<br>
  `loss`: PyTorch module, instantiated train loss class from [losses collection](https://nixtla.github.io/neuralforecast/losses.pytorch.html).<br>
  `valid_loss`: PyTorch module=`loss`, instantiated valid loss class from [losses collection](https://nixtla.github.io/neuralforecast/losses.pytorch.html).<br>
  `max_steps`: int=1000, maximum number of training steps.<br>
  `learning_rate`: float=1e-3, Learning rate between (0, 1).<br>
  `num_lr_decays`: int=-1, Number of learning rate decays, evenly distributed across max_steps.<br>
  `early_stop_patience_steps`: int=-1, Number of validation iterations before early stopping.<br>
  `val_check_steps`: int=100, Number of training steps between every validation loss check.<br>
  `batch_size`: int=32, number of different series in each batch.<br>
  `valid_batch_size`: int=None, number of different series in each validation and test batch, if None uses batch_size.<br>
  `windows_batch_size`: int=1024, number of windows to sample in each training batch, default uses all.<br>
  `inference_windows_batch_size`: int=1024, number of windows to sample in each inference batch.<br>
  `start_padding_enabled`: bool=False, if True, the model will pad the time series with zeros at the beginning, by input size.<br>
  `step_size`: int=1, step size between each window of temporal data.<br>
  `scaler_type`: str='identity', type of scaler for temporal inputs normalization see [temporal scalers](https://nixtla.github.io/neuralforecast/common.scalers.html).<br>
  `random_seed`: int, random_seed for pytorch initializer and numpy generators.<br>
  `num_workers_loader`: int=os.cpu_count(), workers to be used by `TimeSeriesDataLoader`.<br>
  `drop_last_loader`: bool=False, if True `TimeSeriesDataLoader` drops last non-full batch.<br>
  `alias`: str, optional,  Custom name of the model.<br>
  `**trainer_kwargs`: int,  keyword trainer arguments inherited from [PyTorch Lighning's trainer](https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.trainer.trainer.Trainer.html?highlight=trainer).<br>
  
  **References:**<br>
  -[Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2022). "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers"](https://arxiv.org/pdf/2211.14730.pdf)
class_docstr_tok: ["PatchTST","The","PatchTST","model","is","an","efficient","Transformer-based","model","for","multivariate","time","series","forecasting",".","It","is","based","on","two","key","components",":","-","segmentation","of","time","series","into","windows","(","patches",")","which","are","served","as","input","tokens","to","Transformer","-","channel-independence",",","where","each","channel","contains","a","single","univariate","time","series",".","*","*","Parameters",":","*","*","<","br",">","`","h","`",":","int",",","Forecast","horizon",".","<","br",">","`","input_size","`",":","int",",","autorregresive","inputs","size",",","y=","[","1,2,3,4","]","input_size=2","-",">","y_","[","t-2",":","t","]","=","[","1,2","]",".","<","br",">","`","stat_exog_list","`",":","str","list",",","static","exogenous","columns.","<","br",">","`","hist_exog_list","`",":","str","list",",","historic","exogenous","columns.","<","br",">","`","futr_exog_list","`",":","str","list",",","future","exogenous","columns.","<","br",">","`","exclude_insample_y","`",":","bool=False",",","the","model","skips","the","autoregressive","features","y","[","t-input_size",":","t","]","if","True.","<","br",">","`","encoder_layers","`",":","int",",","number","of","layers","for","encoder.","<","br",">","`","n_heads","`",":","int=16",",","number","of","multi-head","'s","attention.","<","br",">","`","hidden_size","`",":","int=128",",","units","of","embeddings","and","encoders.","<","br",">","`","linear_hidden_size","`",":","int=256",",","units","of","linear","layer.","<","br",">","`","dropout","`",":","float=0.1",",","dropout","rate","for","residual","connection.","<","br",">","`","fc_dropout","`",":","float=0.1",",","dropout","rate","for","linear","layer.","<","br",">","`","head_dropout","`",":","float=0.1",",","dropout","rate","for","Flatten","head","layer.","<","br",">","`","attn_dropout","`",":","float=0.1",",","dropout","rate","for","attention","layer.","<","br",">","`","patch_len","`",":","int=32",",","length","of","patch.","<","br",">","`","stride","`",":","int=16",",","stride","of","patch.","<","br",">","`","revin","`",":","bool=True",",","bool","to","use","RevIn.","<","br",">","`","revin_affine","`",":","bool=False",",","bool","to","use","affine","in","RevIn.","<","br",">","`","revin_substract_last","`",":","bool=False",",","bool","to","use","substract","last","in","RevIn.","<","br",">","`","activation","`",":","str='ReLU","'",",","activation","from","[","'gelu","'",",","'relu","'","]",".","<","br",">","`","res_attention","`",":","bool=False",",","bool","to","use","residual","attention.","<","br",">","`","batch_normalization","`",":","bool=False",",","bool","to","use","batch","normalization.","<","br",">","`","learn_pos_embedding","`",":","bool=True",",","bool","to","learn","positional","embedding.","<","br",">","`","loss","`",":","PyTorch","module",",","instantiated","train","loss","class","from","[","losses","collection","]","(","https",":","\/\/nixtla.github.io\/neuralforecast\/losses.pytorch.html",")",".","<","br",">","`","valid_loss","`",":","PyTorch","module=","`","loss","`",",","instantiated","valid","loss","class","from","[","losses","collection","]","(","https",":","\/\/nixtla.github.io\/neuralforecast\/losses.pytorch.html",")",".","<","br",">","`","max_steps","`",":","int=1000",",","maximum","number","of","training","steps.","<","br",">","`","learning_rate","`",":","float=1e-3",",","Learning","rate","between","(","0",",","1",")",".","<","br",">","`","num_lr_decays","`",":","int=-1",",","Number","of","learning","rate","decays",",","evenly","distributed","across","max_steps.","<","br",">","`","early_stop_patience_steps","`",":","int=-1",",","Number","of","validation","iterations","before","early","stopping.","<","br",">","`","val_check_steps","`",":","int=100",",","Number","of","training","steps","between","every","validation","loss","check.","<","br",">","`","batch_size","`",":","int=32",",","number","of","different","series","in","each","batch.","<","br",">","`","valid_batch_size","`",":","int=None",",","number","of","different","series","in","each","validation","and","test","batch",",","if","None","uses","batch_size.","<","br",">","`","windows_batch_size","`",":","int=1024",",","number","of","windows","to","sample","in","each","training","batch",",","default","uses","all.","<","br",">","`","inference_windows_batch_size","`",":","int=1024",",","number","of","windows","to","sample","in","each","inference","batch.","<","br",">","`","start_padding_enabled","`",":","bool=False",",","if","True",",","the","model","will","pad","the","time","series","with","zeros","at","the","beginning",",","by","input","size.","<","br",">","`","step_size","`",":","int=1",",","step","size","between","each","window","of","temporal","data.","<","br",">","`","scaler_type","`",":","str='identity","'",",","type","of","scaler","for","temporal","inputs","normalization","see","[","temporal","scalers","]","(","https",":","\/\/nixtla.github.io\/neuralforecast\/common.scalers.html",")",".","<","br",">","`","random_seed","`",":","int",",","random_seed","for","pytorch","initializer","and","numpy","generators.","<","br",">","`","num_workers_loader","`",":","int=os.cpu_count","(",")",",","workers","to","be","used","by","`","TimeSeriesDataLoader","`",".","<","br",">","`","drop_last_loader","`",":","bool=False",",","if","True","`","TimeSeriesDataLoader","`","drops","last","non-full","batch.","<","br",">","`","alias","`",":","str",",","optional",",","Custom","name","of","the","model.","<","br",">","`","*","*","trainer_kwargs","`",":","int",",","keyword","trainer","arguments","inherited","from","[","PyTorch","Lighning","'s","trainer","]","(","https",":","\/\/pytorch-lightning.readthedocs.io\/en\/stable\/api\/pytorch_lightning.trainer.trainer.Trainer.html","?","highlight=trainer",")",".","<","br",">","*","*","References",":","*","*","<","br",">","-","[","Nie",",","Y.",",","Nguyen",",","N.","H.",",","Sinthong",",","P.",",","&","Kalagnanam",",","J",".","(","2022",")",".","``","A","Time","Series","is","Worth","64","Words",":","Long-term","Forecasting","with","Transformers","''","]","(","https",":","\/\/arxiv.org\/pdf\/2211.14730.pdf",")"]
code: def __init__(
      self,
      h,
      input_size,
      stat_exog_list=None,
      hist_exog_list=None,
      futr_exog_list=None,
      exclude_insample_y=False,
      encoder_layers: int = 3,
      n_heads: int = 16,
      hidden_size: int = 128,
      linear_hidden_size: int = 256,
      dropout: float = 0.2,
      fc_dropout: float = 0.2,
      head_dropout: float = 0.0,
      attn_dropout: float = 0.0,
      patch_len: int = 16,
      stride: int = 8,
      revin: bool = True,
      revin_affine: bool = False,
      revin_subtract_last: bool = True,
      activation: str = "gelu",
      res_attention: bool = True,
      batch_normalization: bool = False,
      learn_pos_embed: bool = True,
      loss=MAE(),
      valid_loss=None,
      max_steps: int = 5000,
      learning_rate: float = 1e-4,
      num_lr_decays: int = -1,
      early_stop_patience_steps: int = -1,
      val_check_steps: int = 100,
      batch_size: int = 32,
      valid_batch_size: Optional[int] = None,
      windows_batch_size=1024,
      inference_windows_batch_size: int = 1024,
      start_padding_enabled=False,
      step_size: int = 1,
      scaler_type: str = "identity",
      random_seed: int = 1,
      num_workers_loader: int = 0,
      drop_last_loader: bool = False,
      **trainer_kwargs,
  ):
      super(PatchTST, self).__init__(
          h=h,
          input_size=input_size,
          hist_exog_list=hist_exog_list,
          stat_exog_list=stat_exog_list,
          futr_exog_list=futr_exog_list,
          exclude_insample_y=exclude_insample_y,
          loss=loss,
          valid_loss=valid_loss,
          max_steps=max_steps,
          learning_rate=learning_rate,
          num_lr_decays=num_lr_decays,
          early_stop_patience_steps=early_stop_patience_steps,
          val_check_steps=val_check_steps,
          batch_size=batch_size,
          valid_batch_size=valid_batch_size,
          windows_batch_size=windows_batch_size,
          inference_windows_batch_size=inference_windows_batch_size,
          start_padding_enabled=start_padding_enabled,
          step_size=step_size,
          scaler_type=scaler_type,
          num_workers_loader=num_workers_loader,
          drop_last_loader=drop_last_loader,
          random_seed=random_seed,
          **trainer_kwargs,
      )
      # Asserts
      if stat_exog_list is not None:
          raise Exception(
              "PatchTST does not yet support static exogenous variables"
          )
      if futr_exog_list is not None:
          raise Exception(
              "PatchTST does not yet support future exogenous variables"
          )
      if hist_exog_list is not None:
          raise Exception(
              "PatchTST does not yet support historical exogenous variables"
          )
  
      c_out = self.loss.outputsize_multiplier
  
      # Fixed hyperparameters
      c_in = 1  # Always univariate
      padding_patch = "end"  # Padding at the end
      pretrain_head = False  # No pretrained head
      norm = (
          "BatchNorm"  # Use BatchNorm (if batch_normalization is True)
      )
      pe = "zeros"  # Initial zeros for positional encoding
      d_k = None  # Key dimension
      d_v = None  # Value dimension
      store_attn = False  # Store attention weights
      head_type = "flatten"  # Head type
      individual = False  # Separate heads for each time series
      max_seq_len = 1024  # Not used
      key_padding_mask = "auto"  # Not used
      padding_var = None  # Not used
      attn_mask = None  # Not used
  
      self.model = PatchTST_backbone(
          c_in=c_in,
          c_out=c_out,
          input_size=input_size,
          h=h,
          patch_len=patch_len,
          stride=stride,
          max_seq_len=max_seq_len,
          n_layers=encoder_layers,
          hidden_size=hidden_size,
          n_heads=n_heads,
          d_k=d_k,
          d_v=d_v,
          linear_hidden_size=linear_hidden_size,
          norm=norm,
          attn_dropout=attn_dropout,
          dropout=dropout,
          act=activation,
          key_padding_mask=key_padding_mask,
          padding_var=padding_var,
          attn_mask=attn_mask,
          res_attention=res_attention,
          pre_norm=batch_normalization,
          store_attn=store_attn,
          pe=pe,
          learn_pe=learn_pos_embed,
          fc_dropout=fc_dropout,
          head_dropout=head_dropout,
          padding_patch=padding_patch,
          pretrain_head=pretrain_head,
          head_type=head_type,
          individual=individual,
          revin=revin,
          affine=revin_affine,
          subtract_last=revin_subtract_last,
      )
  
imports: import math
  import numpy
  from typing import Optional
  import torch
  import torch.nn
  import torch.nn.functional
  from ..common._base_windows import BaseWindows
  from ..losses.pytorch import MAE
label_desc: Use image node_id 1 to create a new PatchTST object from inherited base classes: BaseWindows with example: obj = PatchTST(h, input_size, stat_exog_list, hist_exog_list, futr_exog_list, exclude_insample_y, encoder_layers, n_heads, hidden_size, linear_hidden_size, dropout, fc_dropout, head_dropout, attn_dropout, patch_len, stride, revin, revin_affine, revin_subtract_last, activation, res_attention, batch_normalization, learn_pos_embed, loss, valid_loss, max_steps, learning_rate, num_lr_decays, early_stop_patience_steps, val_check_steps, batch_size, valid_batch_size, windows_batch_size, inference_windows_batch_size, start_padding_enabled, step_size, scaler_type, random_seed, num_workers_loader, drop_last_loader)