Updating README with clean_up_tokenization_spaces

README.md

@@ -33,7 +33,7 @@ Every model uses its own tokenizer trained on language-specific HPLT data.
 
 [The training statistics of all runs](https://api.wandb.ai/links/ltg/kduj7mjn)
 
-## Example usage
+## Example usage (tested with `transformers==4.46.1` and `tokenizers==0.20.1`)
 
 This model currently needs a custom wrapper from `modeling_ltgbert.py`, you should therefore load the model with `trust_remote_code=True`.
 
@@ -50,7 +50,7 @@ output_p = model(**input_text)
 output_text = torch.where(input_text.input_ids == mask_id, output_p.logits.argmax(-1), input_text.input_ids)
 
 # should output: '[CLS] It's a beautiful place.[SEP]'
-print(tokenizer.decode(output_text[0].tolist()))
+print(tokenizer.decode(output_text[0].tolist(), clean_up_tokenization_spaces=True))
 ```
 
 The following classes are currently implemented: `AutoModel`, `AutoModelMaskedLM`, `AutoModelForSequenceClassification`, `AutoModelForTokenClassification`, `AutoModelForQuestionAnswering` and `AutoModeltForMultipleChoice`.