class_docstr: A stack of Graph Attention (GAT) layers with aggregation of multiple attention heads,
  Eqs 5-6 of the GAT paper https://arxiv.org/abs/1710.10903
  
  To use this class as a Keras model, the features and preprocessed adjacency matrix
  should be supplied using:
  
  - the :class:`.FullBatchNodeGenerator` class for node inference
  - the :class:`.ClusterNodeGenerator` class for scalable/inductive node inference using the Cluster-GCN training procedure (https://arxiv.org/abs/1905.07953)
  - the :class:`.FullBatchLinkGenerator` class for link inference
  
  To have the appropriate preprocessing the generator object should be instantiated
  with the `method='gat'` argument.
  
  Examples:
      Creating a GAT node classification model from an existing :class:`.StellarGraph` object `G`::
  
          generator = FullBatchNodeGenerator(G, method="gat")
          gat = GAT(
                  layer_sizes=[8, 4],
                  activations=["elu","softmax"],
                  attn_heads=8,
                  generator=generator,
                  in_dropout=0.5,
                  attn_dropout=0.5,
              )
          x_inp, predictions = gat.in_out_tensors()
  
  Notes:
    - The inputs are tensors with a batch dimension of 1. These are provided by the         :class:`.FullBatchNodeGenerator` object.
  
    - This does not add self loops to the adjacency matrix, you should preprocess
      the adjacency matrix to add self-loops, using the ``method='gat'`` argument
      of the :class:`.FullBatchNodeGenerator`.
  
    - The nodes provided to the :meth:`.FullBatchNodeGenerator.flow` method are
      used by the final layer to select the predictions for those nodes in order.
      However, the intermediate layers before the final layer order the nodes
      in the same way as the adjacency matrix.
  
  .. seealso::
  
     Examples using GAT:
  
     - `node classification <https://stellargraph.readthedocs.io/en/stable/demos/node-classification/gat-node-classification.html>`__
     - `unsupervised representation learning with Deep Graph Infomax <https://stellargraph.readthedocs.io/en/stable/demos/embeddings/deep-graph-infomax-embeddings.html>`__
     - `interpreting GAT predictions <https://stellargraph.readthedocs.io/en/stable/demos/interpretability/gat-node-link-importance.html>`__
     - `ensemble model for node classification <https://stellargraph.readthedocs.io/en/stable/demos/ensembles/ensemble-node-classification-example.html>`__
  
     Appropriate data generators: :class:`.FullBatchNodeGenerator`, :class:`.FullBatchLinkGenerator`, :class:`.ClusterNodeGenerator`.
  
     Related models:
  
     - Other full-batch models: see the documentation of :class:`.FullBatchNodeGenerator` for a full list
     - :class:`.DeepGraphInfomax` for unsupervised training
  
     :class:`.GraphAttention` and :class:`.GraphAttentionSparse` are the base layers out of which a GAT model is built.
  
  Args:
      layer_sizes (list of int): list of output sizes of GAT layers in the stack. The length of this list defines
          the number of GraphAttention layers in the stack.
      generator (FullBatchNodeGenerator): an instance of FullBatchNodeGenerator class constructed on the graph of interest
      attn_heads (int or list of int): number of attention heads in GraphAttention layers. The options are:
  
          - a single integer: the passed value of ``attn_heads`` will be applied to all GraphAttention layers in the stack, except the last layer (for which the number of attn_heads will be set to 1).
          - a list of integers: elements of the list define the number of attention heads in the corresponding layers in the stack.
  
      attn_heads_reduction (list of str or None): reductions applied to output features of each attention head,
          for all layers in the stack. Valid entries in the list are: ``concat``, ``average``.
          If None is passed, the default reductions are applied: ``concat`` reduction to all layers in the stack
          except the final layer, ``average`` reduction to the last layer (Eqs. 5-6 of the GAT paper).
      bias (bool): toggles an optional bias in GAT layers
      in_dropout (float): dropout rate applied to input features of each GAT layer
      attn_dropout (float): dropout rate applied to attention maps
      normalize (str or None): normalization applied to the final output features of the GAT layers stack. Default is None.
      activations (list of str): list of activations applied to each layer's output; defaults to ``['elu', ..., 'elu']``.
      saliency_map_support (bool): If calculating saliency maps using the tools in
          stellargraph.interpretability.saliency_maps this should be True. Otherwise this should be False (default).
      multiplicity (int, optional): The number of nodes to process at a time. This is 1 for a node
          inference and 2 for link inference (currently no others are supported).
      num_nodes (int, optional): The number of nodes in the given graph.
      num_features (int, optional): The dimensions of the node features used as input to the model.
      kernel_initializer (str or func, optional): The initialiser to use for the weights of each layer.
      kernel_regularizer (str or func, optional): The regulariser to use for the weights of each layer.
      kernel_constraint (str or func, optional): The constraint to use for the weights of each layer.
      bias_initializer (str or func, optional): The initialiser to use for the bias of each layer.
      bias_regularizer (str or func, optional): The regulariser to use for the bias of each layer.
      bias_constraint (str or func, optional): The constraint to use for the bias of each layer.
      attn_kernel_initializer (str or func, optional): The initialiser to use for the attention weights.
      attn_kernel_regularizer (str or func, optional): The regulariser to use for the attention weights.
      attn_kernel_constraint (str or func, optional): The constraint to use for the attention bias.
  
  .. note::
      The values for ``multiplicity``, ``num_nodes``, and ``num_features`` are obtained from the
      provided ``generator`` by default. The additional keyword arguments for these parameters
      provide an alternative way to specify them if a generator cannot be supplied.
class_docstr_tok: ["A","stack","of","Graph","Attention","(","GAT",")","layers","with","aggregation","of","multiple","attention","heads",",","Eqs","5-6","of","the","GAT","paper","https",":","\/\/arxiv.org\/abs\/1710.10903","To","use","this","class","as","a","Keras","model",",","the","features","and","preprocessed","adjacency","matrix","should","be","supplied","using",":","-","the",":","class",":","`",".FullBatchNodeGenerator","`","class","for","node","inference","-","the",":","class",":","`",".ClusterNodeGenerator","`","class","for","scalable\/inductive","node","inference","using","the","Cluster-GCN","training","procedure","(","https",":","\/\/arxiv.org\/abs\/1905.07953",")","-","the",":","class",":","`",".FullBatchLinkGenerator","`","class","for","link","inference","To","have","the","appropriate","preprocessing","the","generator","object","should","be","instantiated","with","the","`","method='gat","'","`","argument",".","Examples",":","Creating","a","GAT","node","classification","model","from","an","existing",":","class",":","`",".StellarGraph","`","object","`","G","`",":",":","generator","=","FullBatchNodeGenerator","(","G",",","method=","''","gat","''",")","gat","=","GAT","(","layer_sizes=","[","8",",","4","]",",","activations=","[","``","elu","''",",","''","softmax","''","]",",","attn_heads=8",",","generator=generator",",","in_dropout=0.5",",","attn_dropout=0.5",",",")","x_inp",",","predictions","=","gat.in_out_tensors","(",")","Notes",":","-","The","inputs","are","tensors","with","a","batch","dimension","of","1",".","These","are","provided","by","the",":","class",":","`",".FullBatchNodeGenerator","`","object",".","-","This","does","not","add","self","loops","to","the","adjacency","matrix",",","you","should","preprocess","the","adjacency","matrix","to","add","self-loops",",","using","the","``","method='gat","'","``","argument","of","the",":","class",":","`",".FullBatchNodeGenerator","`",".","-","The","nodes","provided","to","the",":","meth",":","`",".FullBatchNodeGenerator.flow","`","method","are","used","by","the","final","layer","to","select","the","predictions","for","those","nodes","in","order",".","However",",","the","intermediate","layers","before","the","final","layer","order","the","nodes","in","the","same","way","as","the","adjacency","matrix",".","..","seealso",":",":","Examples","using","GAT",":","-","`","node","classification","<","https",":","\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/node-classification\/gat-node-classification.html",">","`","__","-","`","unsupervised","representation","learning","with","Deep","Graph","Infomax","<","https",":","\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/embeddings\/deep-graph-infomax-embeddings.html",">","`","__","-","`","interpreting","GAT","predictions","<","https",":","\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/interpretability\/gat-node-link-importance.html",">","`","__","-","`","ensemble","model","for","node","classification","<","https",":","\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/ensembles\/ensemble-node-classification-example.html",">","`","__","Appropriate","data","generators",":",":","class",":","`",".FullBatchNodeGenerator","`",",",":","class",":","`",".FullBatchLinkGenerator","`",",",":","class",":","`",".ClusterNodeGenerator","`",".","Related","models",":","-","Other","full-batch","models",":","see","the","documentation","of",":","class",":","`",".FullBatchNodeGenerator","`","for","a","full","list","-",":","class",":","`",".DeepGraphInfomax","`","for","unsupervised","training",":","class",":","`",".GraphAttention","`","and",":","class",":","`",".GraphAttentionSparse","`","are","the","base","layers","out","of","which","a","GAT","model","is","built",".","Args",":","layer_sizes","(","list","of","int",")",":","list","of","output","sizes","of","GAT","layers","in","the","stack",".","The","length","of","this","list","defines","the","number","of","GraphAttention","layers","in","the","stack",".","generator","(","FullBatchNodeGenerator",")",":","an","instance","of","FullBatchNodeGenerator","class","constructed","on","the","graph","of","interest","attn_heads","(","int","or","list","of","int",")",":","number","of","attention","heads","in","GraphAttention","layers",".","The","options","are",":","-","a","single","integer",":","the","passed","value","of","``","attn_heads","``","will","be","applied","to","all","GraphAttention","layers","in","the","stack",",","except","the","last","layer","(","for","which","the","number","of","attn_heads","will","be","set","to","1",")",".","-","a","list","of","integers",":","elements","of","the","list","define","the","number","of","attention","heads","in","the","corresponding","layers","in","the","stack",".","attn_heads_reduction","(","list","of","str","or","None",")",":","reductions","applied","to","output","features","of","each","attention","head",",","for","all","layers","in","the","stack",".","Valid","entries","in","the","list","are",":","``","concat","``",",","``","average","``",".","If","None","is","passed",",","the","default","reductions","are","applied",":","``","concat","``","reduction","to","all","layers","in","the","stack","except","the","final","layer",",","``","average","``","reduction","to","the","last","layer","(","Eqs",".","5-6","of","the","GAT","paper",")",".","bias","(","bool",")",":","toggles","an","optional","bias","in","GAT","layers","in_dropout","(","float",")",":","dropout","rate","applied","to","input","features","of","each","GAT","layer","attn_dropout","(","float",")",":","dropout","rate","applied","to","attention","maps","normalize","(","str","or","None",")",":","normalization","applied","to","the","final","output","features","of","the","GAT","layers","stack",".","Default","is","None",".","activations","(","list","of","str",")",":","list","of","activations","applied","to","each","layer","'s","output",";","defaults","to","``","[","'elu","'",",","...",",","'elu","'","]","``",".","saliency_map_support","(","bool",")",":","If","calculating","saliency","maps","using","the","tools","in","stellargraph.interpretability.saliency_maps","this","should","be","True",".","Otherwise","this","should","be","False","(","default",")",".","multiplicity","(","int",",","optional",")",":","The","number","of","nodes","to","process","at","a","time",".","This","is","1","for","a","node","inference","and","2","for","link","inference","(","currently","no","others","are","supported",")",".","num_nodes","(","int",",","optional",")",":","The","number","of","nodes","in","the","given","graph",".","num_features","(","int",",","optional",")",":","The","dimensions","of","the","node","features","used","as","input","to","the","model",".","kernel_initializer","(","str","or","func",",","optional",")",":","The","initialiser","to","use","for","the","weights","of","each","layer",".","kernel_regularizer","(","str","or","func",",","optional",")",":","The","regulariser","to","use","for","the","weights","of","each","layer",".","kernel_constraint","(","str","or","func",",","optional",")",":","The","constraint","to","use","for","the","weights","of","each","layer",".","bias_initializer","(","str","or","func",",","optional",")",":","The","initialiser","to","use","for","the","bias","of","each","layer",".","bias_regularizer","(","str","or","func",",","optional",")",":","The","regulariser","to","use","for","the","bias","of","each","layer",".","bias_constraint","(","str","or","func",",","optional",")",":","The","constraint","to","use","for","the","bias","of","each","layer",".","attn_kernel_initializer","(","str","or","func",",","optional",")",":","The","initialiser","to","use","for","the","attention","weights",".","attn_kernel_regularizer","(","str","or","func",",","optional",")",":","The","regulariser","to","use","for","the","attention","weights",".","attn_kernel_constraint","(","str","or","func",",","optional",")",":","The","constraint","to","use","for","the","attention","bias",".","..","note",":",":","The","values","for","``","multiplicity","``",",","``","num_nodes","``",",","and","``","num_features","``","are","obtained","from","the","provided","``","generator","``","by","default",".","The","additional","keyword","arguments","for","these","parameters","provide","an","alternative","way","to","specify","them","if","a","generator","can","not","be","supplied","."]
code: def __init__(
      self,
      layer_sizes,
      generator=None,
      attn_heads=1,
      attn_heads_reduction=None,
      bias=True,
      in_dropout=0.0,
      attn_dropout=0.0,
      normalize=None,
      activations=None,
      saliency_map_support=False,
      multiplicity=1,
      num_nodes=None,
      num_features=None,
      kernel_initializer="glorot_uniform",
      kernel_regularizer=None,
      kernel_constraint=None,
      bias_initializer="zeros",
      bias_regularizer=None,
      bias_constraint=None,
      attn_kernel_initializer="glorot_uniform",
      attn_kernel_regularizer=None,
      attn_kernel_constraint=None,
  ):
      self.bias = bias
      self.in_dropout = in_dropout
      self.attn_dropout = attn_dropout
      self.generator = generator
      self.saliency_map_support = saliency_map_support
  
      # Check layer_sizes (must be list of int):
      # check type:
      if not isinstance(layer_sizes, list):
          raise TypeError(
              "{}: layer_sizes should be a list of integers; received type {} instead.".format(
                  type(self).__name__, type(layer_sizes).__name__
              )
          )
      # check that values are valid:
      elif not all([isinstance(s, int) and s > 0 for s in layer_sizes]):
          raise ValueError(
              "{}: all elements in layer_sizes should be positive integers!".format(
                  type(self).__name__
              )
          )
      self.layer_sizes = layer_sizes
      n_layers = len(layer_sizes)
  
      # Check attn_heads (must be int or list of int):
      if isinstance(attn_heads, list):
          # check the length
          if not len(attn_heads) == n_layers:
              raise ValueError(
                  "{}: length of attn_heads list ({}) should match the number of GAT layers ({})".format(
                      type(self).__name__, len(attn_heads), n_layers
                  )
              )
          # check that values in the list are valid
          if not all(
              [isinstance(a, int) and a > 0 for a in attn_heads]
          ):
              raise ValueError(
                  "{}: all elements in attn_heads should be positive integers!".format(
                      type(self).__name__
                  )
              )
          self.attn_heads = (
              attn_heads  # (list of int as passed by the user)
          )
  
      elif isinstance(attn_heads, int):
          self.attn_heads = list()
          for l, _ in enumerate(layer_sizes):
              # number of attention heads for layer l: attn_heads (int) for all but the last layer (for which it's set to 1)
              self.attn_heads.append(
                  attn_heads if l < n_layers - 1 else 1
              )
  
      else:
          raise TypeError(
              "{}: attn_heads should be an integer or a list of integers!".format(
                  type(self).__name__
              )
          )
  
      # Check attn_heads_reduction (list of str, or None):
      if attn_heads_reduction is None:
          # set default head reductions, see eqs 5-6 of the GAT paper
          self.attn_heads_reduction = ["concat"] * (n_layers - 1) + [
              "average"
          ]
      else:
          # user-specified list of head reductions (valid entries are 'concat' and 'average')
          # check type (must be a list of str):
          if not isinstance(attn_heads_reduction, list):
              raise TypeError(
                  "{}: attn_heads_reduction should be a string; received type {} instead.".format(
                      type(self).__name__,
                      type(attn_heads_reduction).__name__,
                  )
              )
  
          # check length of attn_heads_reduction list:
          if not len(attn_heads_reduction) == len(layer_sizes):
              raise ValueError(
                  "{}: length of attn_heads_reduction list ({}) should match the number of GAT layers ({})".format(
                      type(self).__name__,
                      len(attn_heads_reduction),
                      n_layers,
                  )
              )
  
          # check that list elements are valid:
          if all(
              [
                  ahr.lower() in {"concat", "average"}
                  for ahr in attn_heads_reduction
              ]
          ):
              self.attn_heads_reduction = attn_heads_reduction
          else:
              raise ValueError(
                  "{}: elements of attn_heads_reduction list should be either 'concat' or 'average'!".format(
                      type(self).__name__
                  )
              )
  
      # Check activations (list of str):
      # check type:
      if activations is None:
          activations = ["elu"] * n_layers
      if not isinstance(activations, list):
          raise TypeError(
              "{}: activations should be a list of strings; received {} instead".format(
                  type(self).__name__, type(activations).__name__
              )
          )
      # check length:
      if not len(activations) == n_layers:
          raise ValueError(
              "{}: length of activations list ({}) should match the number of GAT layers ({})".format(
                  type(self).__name__, len(activations), n_layers
              )
          )
      self.activations = activations
  
      # Check generator and configure sparse adjacency matrix
      if generator is None:
          self.use_sparse = False
          self.multiplicity = _require_without_generator(
              multiplicity, "multiplicity"
          )
          self.n_nodes = _require_without_generator(
              num_nodes, "num_nodes"
          )
          self.n_features = _require_without_generator(
              num_features, "num_features"
          )
      else:
          if not isinstance(
              generator, (FullBatchGenerator, ClusterNodeGenerator)
          ):
              raise TypeError(
                  f"Generator should be a instance of FullBatchNodeGenerator, "
                  f"FullBatchLinkGenerator or ClusterNodeGenerator"
              )
  
          # Copy required information from generator
          self.use_sparse = generator.use_sparse
          self.multiplicity = generator.multiplicity
          self.n_features = generator.features.shape[1]
          if isinstance(generator, FullBatchGenerator):
              self.n_nodes = generator.features.shape[0]
          else:
              self.n_nodes = None
  
      # Set the normalization layer used in the model
      if normalize == "l2":
          self._normalization = Lambda(
              lambda x: K.l2_normalize(x, axis=2)
          )
  
      elif normalize is None or str(normalize).lower() in {
          "none",
          "linear",
      }:
          self._normalization = Lambda(lambda x: x)
  
      else:
          raise ValueError(
              "Normalization should be either 'l2' or None (also allowed as 'none'); received '{}'".format(
                  normalize
              )
          )
  
      # Switch between sparse or dense model
      if self.use_sparse:
          self._gat_layer = GraphAttentionSparse
      else:
          self._gat_layer = GraphAttention
  
      # Initialize a stack of GAT layers
      self._layers = []
      n_layers = len(self.layer_sizes)
      for ii in range(n_layers):
          # Dropout on input node features before each GAT layer
          self._layers.append(Dropout(self.in_dropout))
  
          # GraphAttention layer
          self._layers.append(
              self._gat_layer(
                  units=self.layer_sizes[ii],
                  attn_heads=self.attn_heads[ii],
                  attn_heads_reduction=self.attn_heads_reduction[ii],
                  in_dropout_rate=self.in_dropout,
                  attn_dropout_rate=self.attn_dropout,
                  activation=self.activations[ii],
                  use_bias=self.bias,
                  saliency_map_support=self.saliency_map_support,
                  kernel_initializer=kernel_initializer,
                  kernel_regularizer=kernel_regularizer,
                  kernel_constraint=kernel_constraint,
                  bias_initializer=bias_initializer,
                  bias_regularizer=bias_regularizer,
                  bias_constraint=bias_constraint,
                  attn_kernel_initializer=attn_kernel_initializer,
                  attn_kernel_regularizer=attn_kernel_regularizer,
                  attn_kernel_constraint=attn_kernel_constraint,
              )
          )
  
imports: import warnings
  import tensorflow
  from tensorflow.keras import backend
  from tensorflow.keras import activations, constraints, initializers, regularizers
  from tensorflow.keras.layers import Input, Layer, Dropout, LeakyReLU, Lambda, Reshape
  from ..mapper import FullBatchNodeGenerator, FullBatchGenerator, ClusterNodeGenerator
  from .misc import SqueezedSparseConversion, deprecated_model_function, GatherIndices
label_desc: Use image node_id 1 to create a new GAT object  with example: obj = GAT(layer_sizes, generator, attn_heads, attn_heads_reduction, bias, in_dropout, attn_dropout, normalize, activations, saliency_map_support, multiplicity, num_nodes, num_features, kernel_initializer, kernel_regularizer, kernel_constraint, bias_initializer, bias_regularizer, bias_constraint, attn_kernel_initializer, attn_kernel_regularizer, attn_kernel_constraint)