Tokenization with multiple workers is too slow
[This](https://huggingface.co/docs/datasets/nlp_process#map) is the most performant way to tokenize a dataset (`batched=True, num_proc=None, return_tensors="np"`) 

If`tokenizer.is_fast` returns `True`, `num_proc` must be `None/1` to benefit from the fast tokenizers' parallelism (the fast tokenizers are implemented in Rust, and Rust multi-threading doesn't work well with Python multi-processing)
I am trying to tokenize a few million documents with multiple workers but the tokenization process is taking forever. 

Code snippet:

```
raw_datasets.map(
            encode_function,
            batched=False,
            num_proc=args.preprocessing_num_workers,
            load_from_cache_file=not args.overwrite_cache,
            remove_columns=[name for name in raw_datasets["train"].column_names if name not in ["input_ids", "labels", "attention_mask"]],
            desc="Tokenizing data",
        )
```

Details:

```
transformers==4.28.0.dev0
datasets==4.28.0.dev0
preprocessing_num_workers==48
```
tokenizer == decapoda-research/llama-7b-hf


