
# Wav2Vec2-Large-XLSR-53-German-GPT2

This is an encoder-decoder model for automatic speech recognition trained on on the
MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - DE dataset. The encoder was initialized from
[jonatasgrosman/wav2vec2-large-xlsr-53-german](https://huggingface.co/jonatasgrosman/wav2vec2-large-xlsr-53-german) and
the decoder from [dbmdz/german-gpt2](https://huggingface.co/dbmdz/german-gpt2).

It was trained using a two step process:
* fine-tuning only the cross-attention weights and the decoder using the pre-computed outputs of the Wav2Vec-Modell
  * relatively fast training
  * also works on small GPU (eg. 8 GB)
  * but may take a lot of disk space
  * should already yield decent results
* fine-tuning the model end-to-end
  * much slower
  * needs a bigger GPU

There is also one trick, which seemed to improve performance significantly: adding position embeddings to the
encoder outputs and initializing them with the pre-trained position embeddings of the GPT2 model (See `eval.py`).

The training notebooks are still early drafts. Also results can probably improved a lot by using for example a learning
rate schedule.
{"language": ["de"], "license": "apache-2.0", "tags": ["automatic-speech-recognition", "de", "hf-asr-leaderboard", "mozilla-foundation/common_voice_7_0", "robust-speech-event"], "datasets": ["mozilla-foundation/common_voice_7_0"], "model-index": [{"name": "Wav2Vec2-Large-XLSR-53-German-GPT2", "results": [{"task": {"type": "automatic-speech-recognition", "name": "Automatic Speech Recognition"}, "dataset": {"name": "Common Voice 7", "type": "mozilla-foundation/common_voice_7_0", "args": "de"}, "metrics": [{"type": "wer", "value": 10.02, "name": "Test WER"}, {"type": "cer", "value": 4.7, "name": "Test CER"}]}]}]}
transformers pytorch speech-encoder-decoder automatic-speech-recognition de hf-asr-leaderboard mozilla-foundation/common_voice_7_0 robust-speech-event dataset:mozilla-foundation/common_voice_7_0 license:apache-2.0 model-index endpoints_compatible region:us
TAGS
#transformers #pytorch #speech-encoder-decoder #automatic-speech-recognition #de #hf-asr-leaderboard #mozilla-foundation/common_voice_7_0 #robust-speech-event #dataset-mozilla-foundation/common_voice_7_0 #license-apache-2.0 #model-index #endpoints_compatible #region-us 


# Wav2Vec2-Large-XLSR-53-German-GPT2

This is an encoder-decoder model for automatic speech recognition trained on on the
MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - DE dataset. The encoder was initialized from
jonatasgrosman/wav2vec2-large-xlsr-53-german and
the decoder from dbmdz/german-gpt2.

It was trained using a two step process:
* fine-tuning only the cross-attention weights and the decoder using the pre-computed outputs of the Wav2Vec-Modell
  * relatively fast training
  * also works on small GPU (eg. 8 GB)
  * but may take a lot of disk space
  * should already yield decent results
* fine-tuning the model end-to-end
  * much slower
  * needs a bigger GPU

There is also one trick, which seemed to improve performance significantly: adding position embeddings to the
encoder outputs and initializing them with the pre-trained position embeddings of the GPT2 model (See 'URL').

The training notebooks are still early drafts. Also results can probably improved a lot by using for example a learning
rate schedule.
# Wav2Vec2-Large-XLSR-53-German-GPT2

This is an encoder-decoder model for automatic speech recognition trained on on the
MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - DE dataset. The encoder was initialized from
jonatasgrosman/wav2vec2-large-xlsr-53-german and
the decoder from dbmdz/german-gpt2.

It was trained using a two step process:
* fine-tuning only the cross-attention weights and the decoder using the pre-computed outputs of the Wav2Vec-Modell
  * relatively fast training
  * also works on small GPU (eg. 8 GB)
  * but may take a lot of disk space
  * should already yield decent results
* fine-tuning the model end-to-end
  * much slower
  * needs a bigger GPU

There is also one trick, which seemed to improve performance significantly: adding position embeddings to the
encoder outputs and initializing them with the pre-trained position embeddings of the GPT2 model (See 'URL').

The training notebooks are still early drafts. Also results can probably improved a lot by using for example a learning
rate schedule.
TAGS
#transformers #pytorch #speech-encoder-decoder #automatic-speech-recognition #de #hf-asr-leaderboard #mozilla-foundation/common_voice_7_0 #robust-speech-event #dataset-mozilla-foundation/common_voice_7_0 #license-apache-2.0 #model-index #endpoints_compatible #region-us 
 # Wav2Vec2-Large-XLSR-53-German-GPT2

This is an encoder-decoder model for automatic speech recognition trained on on the
MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - DE dataset. The encoder was initialized from
jonatasgrosman/wav2vec2-large-xlsr-53-german and
the decoder from dbmdz/german-gpt2.

It was trained using a two step process:
* fine-tuning only the cross-attention weights and the decoder using the pre-computed outputs of the Wav2Vec-Modell
  * relatively fast training
  * also works on small GPU (eg. 8 GB)
  * but may take a lot of disk space
  * should already yield decent results
* fine-tuning the model end-to-end
  * much slower
  * needs a bigger GPU

There is also one trick, which seemed to improve performance significantly: adding position embeddings to the
encoder outputs and initializing them with the pre-trained position embeddings of the GPT2 model (See 'URL').

The training notebooks are still early drafts. Also results can probably improved a lot by using for example a learning
rate schedule.
passage: TAGS
#transformers #pytorch #speech-encoder-decoder #automatic-speech-recognition #de #hf-asr-leaderboard #mozilla-foundation/common_voice_7_0 #robust-speech-event #dataset-mozilla-foundation/common_voice_7_0 #license-apache-2.0 #model-index #endpoints_compatible #region-us 
# Wav2Vec2-Large-XLSR-53-German-GPT2

This is an encoder-decoder model for automatic speech recognition trained on on the
MOZILLA-FOUNDATION/COMMON_VOICE_7_0 - DE dataset. The encoder was initialized from
jonatasgrosman/wav2vec2-large-xlsr-53-german and
the decoder from dbmdz/german-gpt2.

It was trained using a two step process:
* fine-tuning only the cross-attention weights and the decoder using the pre-computed outputs of the Wav2Vec-Modell
  * relatively fast training
  * also works on small GPU (eg. 8 GB)
  * but may take a lot of disk space
  * should already yield decent results
* fine-tuning the model end-to-end
  * much slower
  * needs a bigger GPU

There is also one trick, which seemed to improve performance significantly: adding position embeddings to the
encoder outputs and initializing them with the pre-trained position embeddings of the GPT2 model (See 'URL').

The training notebooks are still early drafts. Also results can probably improved a lot by using for example a learning
rate schedule.