class_docstr: Informer
  
      The Informer model tackles the vanilla Transformer computational complexity challenges for long-horizon forecasting.
      The architecture has three distinctive features:
      1) A ProbSparse self-attention mechanism with an O time and memory complexity Llog(L).
      2) A self-attention distilling process that prioritizes attention and efficiently handles long input sequences.
      3) An MLP multi-step decoder that predicts long time-series sequences in a single forward operation rather than step-by-step.
  
  The Informer model utilizes a three-component approach to define its embedding:
      1) It employs encoded autoregressive features obtained from a convolution network.
      2) It uses window-relative positional embeddings derived from harmonic functions.
      3) Absolute positional embeddings obtained from calendar features are utilized.
  
  *Parameters:*<br>
  `h`: int, forecast horizon.<br>
  `input_size`: int, maximum sequence length for truncated train backpropagation. Default -1 uses all history.<br>
  `futr_exog_list`: str list, future exogenous columns.<br>
  `hist_exog_list`: str list, historic exogenous columns.<br>
  `stat_exog_list`: str list, static exogenous columns.<br>
  `exclude_insample_y`: bool=False, the model skips the autoregressive features y[t-input_size:t] if True.<br>
      `decoder_input_size_multiplier`: float = 0.5, .<br>
  `hidden_size`: int=128, units of embeddings and encoders.<br>
  `n_head`: int=4, controls number of multi-head's attention.<br>
  `dropout`: float (0, 1), dropout throughout Informer architecture.<br>
      `factor`: int=3, Probsparse attention factor.<br>
      `conv_hidden_size`: int=32, channels of the convolutional encoder.<br>
      `activation`: str=`GELU`, activation from ['ReLU', 'Softplus', 'Tanh', 'SELU', 'LeakyReLU', 'PReLU', 'Sigmoid', 'GELU'].<br>
  `encoder_layers`: int=2, number of layers for the TCN encoder.<br>
  `decoder_layers`: int=1, number of layers for the MLP decoder.<br>
  `distil`: bool = True, wether the Informer decoder uses bottlenecks.<br>
  `loss`: PyTorch module, instantiated train loss class from [losses collection](https://nixtla.github.io/neuralforecast/losses.pytorch.html).<br>
  `max_steps`: int=1000, maximum number of training steps.<br>
  `learning_rate`: float=1e-3, Learning rate between (0, 1).<br>
  `num_lr_decays`: int=-1, Number of learning rate decays, evenly distributed across max_steps.<br>
  `early_stop_patience_steps`: int=-1, Number of validation iterations before early stopping.<br>
  `val_check_steps`: int=100, Number of training steps between every validation loss check.<br>
  `batch_size`: int=32, number of different series in each batch.<br>
  `valid_batch_size`: int=None, number of different series in each validation and test batch, if None uses batch_size.<br>
  `windows_batch_size`: int=1024, number of windows to sample in each training batch, default uses all.<br>
  `inference_windows_batch_size`: int=1024, number of windows to sample in each inference batch.<br>
  `start_padding_enabled`: bool=False, if True, the model will pad the time series with zeros at the beginning, by input size.<br>
  `scaler_type`: str='robust', type of scaler for temporal inputs normalization see [temporal scalers](https://nixtla.github.io/neuralforecast/common.scalers.html).<br>
  `random_seed`: int=1, random_seed for pytorch initializer and numpy generators.<br>
  `num_workers_loader`: int=os.cpu_count(), workers to be used by `TimeSeriesDataLoader`.<br>
  `drop_last_loader`: bool=False, if True `TimeSeriesDataLoader` drops last non-full batch.<br>
  `alias`: str, optional,  Custom name of the model.<br>
  `**trainer_kwargs`: int,  keyword trainer arguments inherited from [PyTorch Lighning's trainer](https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.trainer.trainer.Trainer.html?highlight=trainer).<br>
  
      *References*<br>
      - [Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, Wancai Zhang. "Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting"](https://arxiv.org/abs/2012.07436)<br>
class_docstr_tok: ["Informer","The","Informer","model","tackles","the","vanilla","Transformer","computational","complexity","challenges","for","long-horizon","forecasting",".","The","architecture","has","three","distinctive","features",":","1",")","A","ProbSparse","self-attention","mechanism","with","an","O","time","and","memory","complexity","Llog","(","L",")",".","2",")","A","self-attention","distilling","process","that","prioritizes","attention","and","efficiently","handles","long","input","sequences",".","3",")","An","MLP","multi-step","decoder","that","predicts","long","time-series","sequences","in","a","single","forward","operation","rather","than","step-by-step",".","The","Informer","model","utilizes","a","three-component","approach","to","define","its","embedding",":","1",")","It","employs","encoded","autoregressive","features","obtained","from","a","convolution","network",".","2",")","It","uses","window-relative","positional","embeddings","derived","from","harmonic","functions",".","3",")","Absolute","positional","embeddings","obtained","from","calendar","features","are","utilized",".","*","Parameters",":","*","<","br",">","`","h","`",":","int",",","forecast","horizon.","<","br",">","`","input_size","`",":","int",",","maximum","sequence","length","for","truncated","train","backpropagation",".","Default","-1","uses","all","history.","<","br",">","`","futr_exog_list","`",":","str","list",",","future","exogenous","columns.","<","br",">","`","hist_exog_list","`",":","str","list",",","historic","exogenous","columns.","<","br",">","`","stat_exog_list","`",":","str","list",",","static","exogenous","columns.","<","br",">","`","exclude_insample_y","`",":","bool=False",",","the","model","skips","the","autoregressive","features","y","[","t-input_size",":","t","]","if","True.","<","br",">","`","decoder_input_size_multiplier","`",":","float","=","0.5",",",".","<","br",">","`","hidden_size","`",":","int=128",",","units","of","embeddings","and","encoders.","<","br",">","`","n_head","`",":","int=4",",","controls","number","of","multi-head","'s","attention.","<","br",">","`","dropout","`",":","float","(","0",",","1",")",",","dropout","throughout","Informer","architecture.","<","br",">","`","factor","`",":","int=3",",","Probsparse","attention","factor.","<","br",">","`","conv_hidden_size","`",":","int=32",",","channels","of","the","convolutional","encoder.","<","br",">","`","activation","`",":","str=","`","GELU","`",",","activation","from","[","'ReLU","'",",","'Softplus","'",",","'Tanh","'",",","'SELU","'",",","'LeakyReLU","'",",","'PReLU","'",",","'Sigmoid","'",",","'GELU","'","]",".","<","br",">","`","encoder_layers","`",":","int=2",",","number","of","layers","for","the","TCN","encoder.","<","br",">","`","decoder_layers","`",":","int=1",",","number","of","layers","for","the","MLP","decoder.","<","br",">","`","distil","`",":","bool","=","True",",","wether","the","Informer","decoder","uses","bottlenecks.","<","br",">","`","loss","`",":","PyTorch","module",",","instantiated","train","loss","class","from","[","losses","collection","]","(","https",":","\/\/nixtla.github.io\/neuralforecast\/losses.pytorch.html",")",".","<","br",">","`","max_steps","`",":","int=1000",",","maximum","number","of","training","steps.","<","br",">","`","learning_rate","`",":","float=1e-3",",","Learning","rate","between","(","0",",","1",")",".","<","br",">","`","num_lr_decays","`",":","int=-1",",","Number","of","learning","rate","decays",",","evenly","distributed","across","max_steps.","<","br",">","`","early_stop_patience_steps","`",":","int=-1",",","Number","of","validation","iterations","before","early","stopping.","<","br",">","`","val_check_steps","`",":","int=100",",","Number","of","training","steps","between","every","validation","loss","check.","<","br",">","`","batch_size","`",":","int=32",",","number","of","different","series","in","each","batch.","<","br",">","`","valid_batch_size","`",":","int=None",",","number","of","different","series","in","each","validation","and","test","batch",",","if","None","uses","batch_size.","<","br",">","`","windows_batch_size","`",":","int=1024",",","number","of","windows","to","sample","in","each","training","batch",",","default","uses","all.","<","br",">","`","inference_windows_batch_size","`",":","int=1024",",","number","of","windows","to","sample","in","each","inference","batch.","<","br",">","`","start_padding_enabled","`",":","bool=False",",","if","True",",","the","model","will","pad","the","time","series","with","zeros","at","the","beginning",",","by","input","size.","<","br",">","`","scaler_type","`",":","str='robust","'",",","type","of","scaler","for","temporal","inputs","normalization","see","[","temporal","scalers","]","(","https",":","\/\/nixtla.github.io\/neuralforecast\/common.scalers.html",")",".","<","br",">","`","random_seed","`",":","int=1",",","random_seed","for","pytorch","initializer","and","numpy","generators.","<","br",">","`","num_workers_loader","`",":","int=os.cpu_count","(",")",",","workers","to","be","used","by","`","TimeSeriesDataLoader","`",".","<","br",">","`","drop_last_loader","`",":","bool=False",",","if","True","`","TimeSeriesDataLoader","`","drops","last","non-full","batch.","<","br",">","`","alias","`",":","str",",","optional",",","Custom","name","of","the","model.","<","br",">","`","*","*","trainer_kwargs","`",":","int",",","keyword","trainer","arguments","inherited","from","[","PyTorch","Lighning","'s","trainer","]","(","https",":","\/\/pytorch-lightning.readthedocs.io\/en\/stable\/api\/pytorch_lightning.trainer.trainer.Trainer.html","?","highlight=trainer",")",".","<","br",">","*","References","*","<","br",">","-","[","Haoyi","Zhou",",","Shanghang","Zhang",",","Jieqi","Peng",",","Shuai","Zhang",",","Jianxin","Li",",","Hui","Xiong",",","Wancai","Zhang",".","``","Informer",":","Beyond","Efficient","Transformer","for","Long","Sequence","Time-Series","Forecasting","''","]","(","https",":","\/\/arxiv.org\/abs\/2012.07436",")","<","br",">"]
code: def __init__(
      self,
      h: int,
      input_size: int,
      stat_exog_list=None,
      hist_exog_list=None,
      futr_exog_list=None,
      exclude_insample_y=False,
      decoder_input_size_multiplier: float = 0.5,
      hidden_size: int = 128,
      dropout: float = 0.05,
      factor: int = 3,
      n_head: int = 4,
      conv_hidden_size: int = 32,
      activation: str = "gelu",
      encoder_layers: int = 2,
      decoder_layers: int = 1,
      distil: bool = True,
      loss=MAE(),
      valid_loss=None,
      max_steps: int = 5000,
      learning_rate: float = 1e-4,
      num_lr_decays: int = -1,
      early_stop_patience_steps: int = -1,
      val_check_steps: int = 100,
      batch_size: int = 32,
      valid_batch_size: Optional[int] = None,
      windows_batch_size=1024,
      inference_windows_batch_size=1024,
      start_padding_enabled=False,
      step_size: int = 1,
      scaler_type: str = "identity",
      random_seed: int = 1,
      num_workers_loader: int = 0,
      drop_last_loader: bool = False,
      **trainer_kwargs,
  ):
      super(Informer, self).__init__(
          h=h,
          input_size=input_size,
          hist_exog_list=hist_exog_list,
          stat_exog_list=stat_exog_list,
          futr_exog_list=futr_exog_list,
          exclude_insample_y=exclude_insample_y,
          loss=loss,
          valid_loss=valid_loss,
          max_steps=max_steps,
          learning_rate=learning_rate,
          num_lr_decays=num_lr_decays,
          early_stop_patience_steps=early_stop_patience_steps,
          val_check_steps=val_check_steps,
          batch_size=batch_size,
          valid_batch_size=valid_batch_size,
          windows_batch_size=windows_batch_size,
          inference_windows_batch_size=inference_windows_batch_size,
          start_padding_enabled=start_padding_enabled,
          step_size=step_size,
          scaler_type=scaler_type,
          num_workers_loader=num_workers_loader,
          drop_last_loader=drop_last_loader,
          random_seed=random_seed,
          **trainer_kwargs,
      )
  
      # Architecture
      self.futr_input_size = len(self.futr_exog_list)
      self.hist_input_size = len(self.hist_exog_list)
      self.stat_input_size = len(self.stat_exog_list)
  
      if self.stat_input_size > 0:
          raise Exception(
              "Informer does not support static variables yet"
          )
  
      if self.hist_input_size > 0:
          raise Exception(
              "Informer does not support historical variables yet"
          )
  
      self.label_len = int(
          np.ceil(input_size * decoder_input_size_multiplier)
      )
      if (self.label_len >= input_size) or (self.label_len <= 0):
          raise Exception(
              f"Check decoder_input_size_multiplier={decoder_input_size_multiplier}, range (0,1)"
          )
  
      if activation not in ["relu", "gelu"]:
          raise Exception(f"Check activation={activation}")
  
      self.c_out = self.loss.outputsize_multiplier
      self.output_attention = False
      self.enc_in = 1
      self.dec_in = 1
  
      # Embedding
      self.enc_embedding = DataEmbedding(
          c_in=self.enc_in,
          exog_input_size=self.hist_input_size,
          hidden_size=hidden_size,
          pos_embedding=True,
          dropout=dropout,
      )
      self.dec_embedding = DataEmbedding(
          self.dec_in,
          exog_input_size=self.hist_input_size,
          hidden_size=hidden_size,
          pos_embedding=True,
          dropout=dropout,
      )
  
      # Encoder
      self.encoder = TransEncoder(
          [
              TransEncoderLayer(
                  AttentionLayer(
                      ProbAttention(
                          False,
                          factor,
                          attention_dropout=dropout,
                          output_attention=self.output_attention,
                      ),
                      hidden_size,
                      n_head,
                  ),
                  hidden_size,
                  conv_hidden_size,
                  dropout=dropout,
                  activation=activation,
              )
              for l in range(encoder_layers)
          ],
          [ConvLayer(hidden_size) for l in range(encoder_layers - 1)]
          if distil
          else None,
          norm_layer=torch.nn.LayerNorm(hidden_size),
      )
      # Decoder
      self.decoder = TransDecoder(
          [
              TransDecoderLayer(
                  AttentionLayer(
                      ProbAttention(
                          True,
                          factor,
                          attention_dropout=dropout,
                          output_attention=False,
                      ),
                      hidden_size,
                      n_head,
                  ),
                  AttentionLayer(
                      ProbAttention(
                          False,
                          factor,
                          attention_dropout=dropout,
                          output_attention=False,
                      ),
                      hidden_size,
                      n_head,
                  ),
                  hidden_size,
                  conv_hidden_size,
                  dropout=dropout,
                  activation=activation,
              )
              for l in range(decoder_layers)
          ],
          norm_layer=torch.nn.LayerNorm(hidden_size),
          projection=nn.Linear(hidden_size, self.c_out, bias=True),
      )
  
imports: import math
  import numpy
  from typing import Optional
  import torch
  import torch.nn
  from neuralforecast.common._modules import TransEncoderLayer, TransEncoder, TransDecoderLayer, TransDecoder, DataEmbedding, AttentionLayer
  from ..common._base_windows import BaseWindows
  from ..losses.pytorch import MAE
label_desc: Use image node_id 1 to create a new Informer object from inherited base classes: BaseWindows with example: obj = Informer(h, input_size, stat_exog_list, hist_exog_list, futr_exog_list, exclude_insample_y, decoder_input_size_multiplier, hidden_size, dropout, factor, n_head, conv_hidden_size, activation, encoder_layers, decoder_layers, distil, loss, valid_loss, max_steps, learning_rate, num_lr_decays, early_stop_patience_steps, val_check_steps, batch_size, valid_batch_size, windows_batch_size, inference_windows_batch_size, start_padding_enabled, step_size, scaler_type, random_seed, num_workers_loader, drop_last_loader)