class_docstr: Layer that computes attention strengths by masking out the "future".
  
  Causal attention uses masking to prevent a given sequence position from
  attending to positions greater than / following it. This is used, for
  example, when training autoregressive sequence models, or when decoding a
  sequence symbol by symbol.
  
  This layer performs the core per-head attention calculation. The layer
  assumes that any splitting into attention heads precedes it, and that any
  merging of attention heads will follow it.
class_docstr_tok: ["Layer","that","computes","attention","strengths","by","masking","out","the","``","future","''",".","Causal","attention","uses","masking","to","prevent","a","given","sequence","position","from","attending","to","positions","greater","than","\/","following","it",".","This","is","used",",","for","example",",","when","training","autoregressive","sequence","models",",","or","when","decoding","a","sequence","symbol","by","symbol",".","This","layer","performs","the","core","per-head","attention","calculation",".","The","layer","assumes","that","any","splitting","into","attention","heads","precedes","it",",","and","that","any","merging","of","attention","heads","will","follow","it","."]
docstr: Creates a :py:class:`DotProductCausalAttention` instance.
  
  Args:
    dropout: Probababilistic rate for attention dropout, which overrides
        (sets to zero) some attention strengths derived from query-key
        matching. As a result, on a given forward pass, some value vectors
        don't contribute to the output, analogous to how regular dropout can
        cause some node activations to be ignored. Applies only if layer is
        created in ``'train'`` mode.
    max_inference_length: Maximum sequence length allowed in non-training
        modes.
    mode: One of ``'train'``, ``'eval'``, or ``'predict'``.
code: def __init__(
      self, dropout=0.0, max_inference_length=2048, mode="train"
  ):
      """Creates a :py:class:`DotProductCausalAttention` instance.
  
      Args:
        dropout: Probababilistic rate for attention dropout, which overrides
            (sets to zero) some attention strengths derived from query-key
            matching. As a result, on a given forward pass, some value vectors
            don't contribute to the output, analogous to how regular dropout can
            cause some node activations to be ignored. Applies only if layer is
            created in ``'train'`` mode.
        max_inference_length: Maximum sequence length allowed in non-training
            modes.
        mode: One of ``'train'``, ``'eval'``, or ``'predict'``.
      """
      super().__init__(n_in=3, n_out=1)
      self._dropout = dropout
      self._mode = mode
      self._max_len = max_inference_length
      self._portal_mask = self.monkey_patched_mask()
  
imports: import numpy
  from trax import fastmath
  from trax.fastmath import numpy
  from trax.layers import base
  from trax.layers import combinators
  from trax.layers import convolution
  from trax.layers import core
  from trax.layers import initializers
  from trax.layers.assert_shape import assert_shape
  from trax.layers.base import Fn
  from trax.layers.research import sparsity
label_desc: Use image node_id 1 to create a new DotProductCausalAttention object from inherited base classes: base with example: obj = DotProductCausalAttention(dropout, max_inference_length, mode)