class_docstr: A utility attention mask class that allows one to:
      - Create a causal 4d mask
      - Create a causal 4d mask with slided window
      - Convert a 2d attention mask (batch_size, query_length) to a 4d attention mask (batch_size, 1, query_length,
        key_value_length) that can be multiplied with attention scores
  
  Examples:
  
  ```python
  >>> import torch
  >>> from transformers.modeling_attn_mask_utils import AttentionMaskConverter
  
  >>> converter = AttentionMaskConverter(True)
  >>> converter.to_4d(torch.tensor([[0, 0, 0, 1, 1]]), 5, key_value_length=5, dtype=torch.float32)
  tensor([[[[-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
          [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
          [-3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38, -3.4028e+38],
          [-3.4028e+38, -3.4028e+38, -3.4028e+38,  0.0000e+00, -3.4028e+38],
          [-3.4028e+38, -3.4028e+38, -3.4028e+38,  0.0000e+00,  0.0000e+00]]]])
  ```
  
  Parameters:
      is_causal (`bool`):
          Whether the attention mask should be a uni-directional (causal) or bi-directional mask.
  
      sliding_window (`int`, *optional*):
          Optionally, the sliding window masks can be created if `sliding_window` is defined to a positive integer.
class_docstr_tok: ["A","utility","attention","mask","class","that","allows","one","to",":","-","Create","a","causal","4d","mask","-","Create","a","causal","4d","mask","with","slided","window","-","Convert","a","2d","attention","mask","(","batch_size",",","query_length",")","to","a","4d","attention","mask","(","batch_size",",","1",",","query_length",",","key_value_length",")","that","can","be","multiplied","with","attention","scores","Examples",":","``","`","python",">",">",">","import","torch",">",">",">","from","transformers.modeling_attn_mask_utils","import","AttentionMaskConverter",">",">",">","converter","=","AttentionMaskConverter","(","True",")",">",">",">","converter.to_4d","(","torch.tensor","(","[","[","0",",","0",",","0",",","1",",","1","]","]",")",",","5",",","key_value_length=5",",","dtype=torch.float32",")","tensor","(","[","[","[","[","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38","]",",","[","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38","]",",","[","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38","]",",","[","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38",",","0.0000e+00",",","-3.4028e+38","]",",","[","-3.4028e+38",",","-3.4028e+38",",","-3.4028e+38",",","0.0000e+00",",","0.0000e+00","]","]","]","]",")","``","`","Parameters",":","is_causal","(","`","bool","`",")",":","Whether","the","attention","mask","should","be","a","uni-directional","(","causal",")","or","bi-directional","mask",".","sliding_window","(","`","int","`",",","*","optional","*",")",":","Optionally",",","the","sliding","window","masks","can","be","created","if","`","sliding_window","`","is","defined","to","a","positive","integer","."]
docstr: Attend to all tokens in masked rows from the expanded attention mask, for example the relevant first rows when
  using left padding. This is required by F.scaled_dot_product_attention memory-efficient attention path.
  Details: https://github.com/pytorch/pytorch/issues/110213
  
  `expanded_mask` is [bsz, num_masks, tgt_seq_len, src_seq_len] or [bsz, tgt_seq_len, src_seq_len].
  `attention_mask` is [bsz, src_seq_len].
  
  The dimension num_masks of `expanded_mask` is most often 1, but it can also be the number of heads in the case of alibi attention bias.
  
  For example, if `attention_mask` is
  ```
  [[0, 0, 1],
   [1, 1, 1],
   [0, 1, 1]]
  ```
  and `expanded_mask` is (e.g. here left-padding case)
  ```
  [[[[0, 0, 0],
     [0, 0, 0],
     [0, 0, 1]]],
   [[[1, 0, 0],
     [1, 1, 0],
     [1, 1, 1]]],
   [[[0, 0, 0],
     [0, 1, 0],
     [0, 1, 1]]]]
  ```
  then the modified `expanded_mask` will be
  ```
  [[[[1, 1, 1],   <-- modified
     [1, 1, 1],   <-- modified
     [0, 0, 1]]],
   [[[1, 0, 0],
     [1, 1, 0],
     [1, 1, 1]]],
   [[[1, 1, 1],   <-- modified
     [0, 1, 0],
     [0, 1, 1]]]]
  ```
docstr_tok: ["Attend","to","all","tokens","in","masked","rows","from","the","expanded","attention","mask",",","for","example","the","relevant","first","rows","when","using","left","padding",".","This","is","required","by","F.scaled_dot_product_attention","memory-efficient","attention","path",".","Details",":","https",":","\/\/github.com\/pytorch\/pytorch\/issues\/110213","`","expanded_mask","`","is","[","bsz",",","num_masks",",","tgt_seq_len",",","src_seq_len","]","or","[","bsz",",","tgt_seq_len",",","src_seq_len","]",".","`","attention_mask","`","is","[","bsz",",","src_seq_len","]",".","The","dimension","num_masks","of","`","expanded_mask","`","is","most","often","1",",","but","it","can","also","be","the","number","of","heads","in","the","case","of","alibi","attention","bias",".","For","example",",","if","`","attention_mask","`","is","``","`","[","[","0",",","0",",","1","]",",","[","1",",","1",",","1","]",",","[","0",",","1",",","1","]","]","``","`","and","`","expanded_mask","`","is","(","e.g",".","here","left-padding","case",")","``","`","[","[","[","[","0",",","0",",","0","]",",","[","0",",","0",",","0","]",",","[","0",",","0",",","1","]","]","]",",","[","[","[","1",",","0",",","0","]",",","[","1",",","1",",","0","]",",","[","1",",","1",",","1","]","]","]",",","[","[","[","0",",","0",",","0","]",",","[","0",",","1",",","0","]",",","[","0",",","1",",","1","]","]","]","]","``","`","then","the","modified","`","expanded_mask","`","will","be","``","`","[","[","[","[","1",",","1",",","1","]",",","<","--","modified","[","1",",","1",",","1","]",",","<","--","modified","[","0",",","0",",","1","]","]","]",",","[","[","[","1",",","0",",","0","]",",","[","1",",","1",",","0","]",",","[","1",",","1",",","1","]","]","]",",","[","[","[","1",",","1",",","1","]",",","<","--","modified","[","0",",","1",",","0","]",",","[","0",",","1",",","1","]","]","]","]","``","`"]
file_path: transformers/src/transformers/modeling_attn_mask_utils.py
imports: from dataclasses import dataclass
  from typing import List, Optional, Tuple, Union
  import torch
label_desc: Use image node_id 6 for calling the AttentionMaskConverter obj's underlying member method code with example usage: obj._unmask_unattended(expanded_mask, attention_mask, unmasked_value) and returns: expanded_mask, expanded_mask