class_docstr: This lobe enables the integration of HuggingFace
   wav2vec2.0 models to be pretrained.
  
  Source paper: https://arxiv.org/abs/2006.11477
  Transformer from HuggingFace needs to be installed:
  https://huggingface.co/transformers/installation.html
  
  The return is an HuggingFace format and the mask indices that contains:
  https://huggingface.co/transformers/model_doc/wav2vec2.html#wav2vec2forpretraining
  
  For instance, it returns the loss that can be accessed with .loss
  
  Arguments
  ---------
  source : str
      HuggingFace hub name: e.g "facebook/wav2vec2-large-lv60"
  save_path : str
      Path (dir) of the downloaded model.
  mask_prob : float (default: 0.65)
      Probability of masking a given frame. Default is taken from the paper.
  mask_length : float (default: 10)
      Length (i.e. number of consecutive masked frames). Default is taken from
      the paper.
  
  Example
  -------
  >>> inputs = torch.rand([10, 32000])
  >>> model_hub = "facebook/wav2vec2-base-960h"
  >>> save_path = "savedir"
  >>> model = Wav2Vec2Pretrain(model_hub, save_path)
  >>> outputs, _ = model(inputs, wav_lens=None)
class_docstr_tok: ["This","lobe","enables","the","integration","of","HuggingFace","wav2vec2.0","models","to","be","pretrained",".","Source","paper",":","https",":","\/\/arxiv.org\/abs\/2006.11477","Transformer","from","HuggingFace","needs","to","be","installed",":","https",":","\/\/huggingface.co\/transformers\/installation.html","The","return","is","an","HuggingFace","format","and","the","mask","indices","that","contains",":","https",":","\/\/huggingface.co\/transformers\/model_doc\/wav2vec2.html","#","wav2vec2forpretraining","For","instance",",","it","returns","the","loss","that","can","be","accessed","with",".loss","Arguments","--","--","--","--","-","source",":","str","HuggingFace","hub","name",":","e.g","``","facebook\/wav2vec2-large-lv60","''","save_path",":","str","Path","(","dir",")","of","the","downloaded","model",".","mask_prob",":","float","(","default",":","0.65",")","Probability","of","masking","a","given","frame",".","Default","is","taken","from","the","paper",".","mask_length",":","float","(","default",":","10",")","Length","(","i.e",".","number","of","consecutive","masked","frames",")",".","Default","is","taken","from","the","paper",".","Example","--","--","--","-",">",">",">","inputs","=","torch.rand","(","[","10",",","32000","]",")",">",">",">","model_hub","=","``","facebook\/wav2vec2-base-960h","''",">",">",">","save_path","=","``","savedir","''",">",">",">","model","=","Wav2Vec2Pretrain","(","model_hub",",","save_path",")",">",">",">","outputs",",","_","=","model","(","inputs",",","wav_lens=None",")"]
docstr: Takes an input waveform and return its corresponding wav2vec encoding.
  
  Arguments
  ---------
  wav : torch.Tensor (signal)
      A batch of audio signals to transform to features.
  wav_len : tensor
      The relative length of the wav given in SpeechBrain format.
docstr_tok: ["Takes","an","input","waveform","and","return","its","corresponding","wav2vec","encoding",".","Arguments","--","--","--","--","-","wav",":","torch.Tensor","(","signal",")","A","batch","of","audio","signals","to","transform","to","features",".","wav_len",":","tensor","The","relative","length","of","the","wav","given","in","SpeechBrain","format","."]
code: def forward(self, wav, wav_lens=None):
      """Takes an input waveform and return its corresponding wav2vec encoding.
  
      Arguments
      ---------
      wav : torch.Tensor (signal)
          A batch of audio signals to transform to features.
      wav_len : tensor
          The relative length of the wav given in SpeechBrain format.
      """
      batch_size, raw_sequence_length = wav.shape
  
      if self.normalize_wav:
          wav = F.layer_norm(wav, wav.shape)
  
      sequence_length = self.model._get_feat_extract_output_lengths(
          raw_sequence_length
      ).item()
  
      # 1. Compute the indices that will be masked
      mask_time_indices = _compute_mask_indices(
          (batch_size, sequence_length),
          mask_prob=self.mask_prob,
          mask_length=self.mask_length,
      )
      torch_mask_time_indices = torch.tensor(
          mask_time_indices,
          device=wav.device,
          dtype=torch.long,
      )
      padding_mask = make_padding_masks(wav, wav_len=wav_lens)
  
      # 2. Sample the negative samples from the entire sequence.
      # Fairseq does it only on the masked indices, but this only work if you
      # have long sentences. For more versatily, we sample on the entire sequence.
      # value.
      full_sentence_indices = np.ones((batch_size, sequence_length))
  
      # print(np.sum(mask_time_indices, axis=1))
      negative_sample_indices = torch.tensor(
          transformers.models.wav2vec2.modeling_wav2vec2._sample_negative_indices(
              (batch_size, sequence_length),
              num_negatives=self.config.num_negatives,
              mask_time_indices=full_sentence_indices,
          ),
          device=wav.device,
          dtype=torch.long,
      )
  
      return (
          self.model(
              wav,
              mask_time_indices=torch_mask_time_indices,
              sampled_negative_indices=negative_sample_indices,
              attention_mask=padding_mask,
          ),
          torch_mask_time_indices,
      )
  
file_path: speechbrain/speechbrain/lobes/models/huggingface_transformers/wav2vec2.py
imports: import torch
  import logging
  import numpy
  import torch.nn.functional
  from speechbrain.lobes.models.huggingface_transformers.huggingface import HFTransformersInterface
  from speechbrain.lobes.models.huggingface_transformers.huggingface import make_padding_masks
  import transformers
  from transformers.models.wav2vec2.modeling_wav2vec2 import _compute_mask_indices
label_desc: Use image node_id 2 for calling the Wav2Vec2Pretrain obj's underlying member method code with example usage: obj.forward(wav, wav_lens) and returns: self,  torch_mask_time_indices