OSError: Memory mapping file failed: Cannot allocate memory
Hi! This error means that PyArrow's internal [`mmap`](https://man7.org/linux/man-pages/man2/mmap.2.html) call failed to allocate memory, which can be tricky to debug. Since this error is more related to PyArrow than us, I think it's best to report this issue in their [repo](https://github.com/apache/arrow) (they are more experienced on this matter). Also, googling "mmap cannot allocate memory" returns some approaches to solving this problem.
### Describe the bug

Hello, I have a series of datasets each of 5 GB, 600 datasets in total. So together this makes 3TB. 

When I trying to load all the 600 datasets into memory, I get the above error message. 

Is this normal because I'm hitting the max size of memory mapping of the OS? 


Thank you

```terminal
0_21/cache-e9c42499f65b1881.arrow
load_hf_datasets_from_disk:  82%|████████████████████████████████████████████████████████████████████████████████████████████████████▍                     | 494/600 [07:26<01:35,  1.11it/s]
Traceback (most recent call last):
  File "example_load_genkalm_dataset.py", line 35, in <module>
    multi_ds.post_process(max_node_num=args.max_node_num,max_seq_length=args.max_seq_length,delay=args.delay)
  File "/home/geng/GenKaLM/src/dataloader/dataset.py", line 142, in post_process
    genkalm_dataset = GenKaLM_Dataset.from_hf_dataset(path_or_name=ds_path, max_seq_length=self.max_seq_length,
  File "/home/geng/GenKaLM/src/dataloader/dataset.py", line 47, in from_hf_dataset
    hf_ds = load_from_disk(path_or_name)
  File "/home/geng/.conda/envs/genkalm/lib/python3.8/site-packages/datasets/load.py", line 1848, in load_from_disk
    return Dataset.load_from_disk(dataset_path, keep_in_memory=keep_in_memory, storage_options=storage_options)
  File "/home/geng/.conda/envs/genkalm/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1549, in load_from_disk
    arrow_table = concat_tables(
  File "/home/geng/.conda/envs/genkalm/lib/python3.8/site-packages/datasets/table.py", line 1805, in concat_tables
    tables = list(tables)
  File "/home/geng/.conda/envs/genkalm/lib/python3.8/site-packages/datasets/arrow_dataset.py", line 1550, in <genexpr>
    table_cls.from_file(Path(dataset_path, data_file["filename"]).as_posix())
  File "/home/geng/.conda/envs/genkalm/lib/python3.8/site-packages/datasets/table.py", line 1065, in from_file
    table = _memory_mapped_arrow_table_from_file(filename)
  File "/home/geng/.conda/envs/genkalm/lib/python3.8/site-packages/datasets/table.py", line 50, in _memory_mapped_arrow_table_from_file
    memory_mapped_stream = pa.memory_map(filename)
  File "pyarrow/io.pxi", line 950, in pyarrow.lib.memory_map
  File "pyarrow/io.pxi", line 911, in pyarrow.lib.MemoryMappedFile._open
  File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 115, in pyarrow.lib.check_status
OSError: Memory mapping file failed: Cannot allocate memory
```








### Steps to reproduce the bug

Sorry I can not provide a reproducible code as the data is stored on my server and it's too large to share.

### Expected behavior

I expect the 3TB of data can be fully mapped to memory

### Environment info


- `datasets` version: 2.9.0
- Platform: Linux-4.15.0-204-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.6
- PyArrow version: 11.0.0
- Pandas version: 1.0.1