Update README.md

README.md

@@ -1,3 +1,31 @@
 ---
 license: mit
 ---
+
+A GPT2-tokenizer for English and German with a vocabulary size of 88,301.
+
+This tokenizer is created by merging the [original GPT2](https://huggingface.co/gpt2) tokenizer (English) with a [German tokenizer](https://huggingface.co/malteos/gpt2-xl-wechsel-german).
+
+## Steps to reproduce
+
+```python
+from transformers import AutoTokenizer
+
+a_tokenizer = AutoTokenizer.from_pretrained('gpt2')
+b_tokenizer = AutoTokenizer.from_pretrained('malteos/gpt2-xl-wechsel-german')
+
+a_vocab = set(a_tokenizer.vocab.keys())  # len(a_vocab)=50257
+b_vocab = set(b_tokenizer.vocab.keys())  # len(b_vocab)=50257
+
+missing_tokens_in_a = b_vocab - a_vocab  # len = 38044 
+
+a_tokenizer.add_tokens(list(missing_tokens_in_a))
+
+a_tokenizer.save('opengptx-en-de') # len = 88301
+
+
+
+```
+
+
+