File size: 2,031 Bytes
5c623ce 179a858 5c623ce ab7e403 5c623ce ab7e403 5c623ce 2fc3736 5c623ce 2fc3736 5c623ce 2fc3736 5c623ce 179a858 5c623ce 179a858 5c623ce 179a858 5c623ce 2fc3736 5c623ce 2fc3736 5c623ce |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
---
library_name: peft
base_model: openai/whisper-large-v2
datasets:
- mozilla-foundation/common_voice_16_0
language:
- ja
metrics:
- wer
---
# Model Card for Model ID
Japanese transcription, testing in progress to see results, main personal use cases are japanese comedy
usage 9GB vram with this Lora
## Model Details
### Model Description
openai-whisper-large-v2-LORA-ja
- **Developed by:** FZNX
- **Model type:** PEFT LORA
- **Language(s) (NLP):** Fine tune Japanese on whisper common 16
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** Whisper Large V2
## How to Get Started with the Model
import torch
from transformers import (
AutomaticSpeechRecognitionPipeline,
WhisperForConditionalGeneration,
WhisperTokenizer,
WhisperProcessor,
)
from peft import PeftModel, PeftConfig
peft_model_id = "fznx92/openai-whisper-large-v2-ja-transcribe-colab"
sample = "insert mp3 file location here"
language = "japanese"
task = "transcribe"
peft_config = PeftConfig.from_pretrained(peft_model_id)
model = WhisperForConditionalGeneration.from_pretrained(
peft_config.base_model_name_or_path,
)
model = PeftModel.from_pretrained(model, peft_model_id)
model.to("cuda").half()
processor = WhisperProcessor.from_pretrained(peft_config.base_model_name_or_path, language=language, task=task)
pipe = AutomaticSpeechRecognitionPipeline(model=model, tokenizer=processor.tokenizer, feature_extractor=processor.feature_extractor, batch_size=8, torch_dtype=torch.float16, device="cuda:0")
def transcribe(audio, return_timestamps=False):
text = pipe(audio, chunk_length_s=30, return_timestamps=return_timestamps, generate_kwargs={"language": language, "task": task})["text"]
return text
transcript = transcribe(sample)
print(transcript)
### Training Data
Common Voice 16 dataset
### Training Procedure
via Google Colab T5 @ 6 hours
## Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->
### Framework versions
- PEFT 0.7.1 |