Failure to hash function when using .map() 
### Describe the bug

_Parameter 'function'=<function process at 0x7f1ec4388af0> of the transform datasets.arrow_dataset.Dataset.\_map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed._

This issue with `.map()` happens for me consistently, as also described in closed issue #4506

Dataset indices can be individually serialized using dill and pickle without any errors.  I'm using tiktoken to encode in the function passed to map(). Similarly, indices can be individually encoded without error.

### Steps to reproduce the bug

```py
from datasets import load_dataset
import tiktoken

dataset = load_dataset("stas/openwebtext-10k")

enc = tiktoken.get_encoding("gpt2")

tokenized = dataset.map(
    process,
    remove_columns=['text'],
    desc="tokenizing the OWT splits",
)

def process(example):
        ids = enc.encode(example['text'])
        ids.append(enc.eot_token)
        out = {'ids': ids, 'len': len(ids)}
        return out
```

### Expected behavior

Should encode simple text objects.

### Environment info


Python versions tried: both 3.8 and 3.10.10
`PYTHONUTF8=1` as env variable

Datasets tried: 
- stas/openwebtext-10k
- rotten_tomatoes
- local text file

OS: Ubuntu Linux 20.04

Package versions:
- torch 1.13.1
- dill 0.3.4 (if using 0.3.6 - same issue)
- datasets 2.9.0
- tiktoken 0.2.0

