 load JSON files, get the errors
Ok I see, you have the paragraphs so you just need to process them to extract the questions and answers. I think you can process the SQuAD-like data this way:
```python
def process_squad(articles):
    out = {
        "title": [],
        "context": [],
        "question": [],
        "id": [],
        "answers": [],
    }
    for title, paragraphs in zip(articles["title"], articles["paragraphs"]):
        for paragraph in paragraphs:
            for qa in paragraph["qas"]:
                out["title"].append(title)
                out["context"].append(paragraph["context"])
                out["question"].append(qa["question"])
                out["id"].append(qa["id"])
                out["answers"].append({
                    "answer_start": [answer["answer_start"] for answer in qa["answers"]],
                    "text": [answer["text"] for answer in qa["answers"]],
                })
    return out

dataset = dataset.map(process_squad, batched=True, remove_columns=["paragraphs"])
```

I adapted the code from [squad.py](https://github.com/huggingface/datasets/blob/master/datasets/squad/squad.py). The code takes as input a batch of articles (title + paragraphs) and gets all the questions and answers from the JSON structure.

The output is a dataset with `features: ['answers', 'context', 'id', 'question', 'title']`

Let me know if that helps !

Hi, does this bug be fixed? when I load JSON files, I get the same errors by the command 
`!python3 run.py --do_train --task qa --dataset squad-retrain-data/train-v2.0.json --output_dir ./re_trained_model/`

change the dateset to load json by refering to https://huggingface.co/docs/datasets/loading.html
`dataset = datasets.load_dataset('json', data_files=args.dataset)`

Errors:
`Downloading and preparing dataset json/default (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to /root/.cache/huggingface/datasets/json/default-c1e124ad488911b8/0.0.0/45636811569ec4a6630521c18235dfbbab83b7ab572e3393c5ba68ccabe98264...
`

_Originally posted by @yanllearnn in https://github.com/huggingface/datasets/issues/730#issuecomment-981095050_
 load JSON files, get the errors 
 Hi, does this bug be fixed? when I load JSON files, I get the same errors by the command 
`!python3 run.py --do_train --task qa --dataset squad-retrain-data/train-v2.0.json --output_dir ./re_trained_model/`

change the dateset to load json by refering to https://huggingface.co/docs/datasets/loading.html
`dataset = datasets.load_dataset('json', data_files=args.dataset)`

Errors:
`Downloading and preparing dataset json/default (download: Unknown size, generated: Unknown size, post-processed: Unknown size, total: Unknown size) to /root/.cache/huggingface/datasets/json/default-c1e124ad488911b8/0.0.0/45636811569ec4a6630521c18235dfbbab83b7ab572e3393c5ba68ccabe98264...
`

_Originally posted by @yanllearnn in https://github.com/huggingface/datasets/issues/730#issuecomment-981095050_ 
 Ok I see, you have the paragraphs so you just need to process them to extract the questions and answers. I think you can process the SQuAD-like data this way:
```python
def process_squad(articles):
    out = {
        "title": [],
        "context": [],
        "question": [],
        "id": [],
        "answers": [],
    }
    for title, paragraphs in zip(articles["title"], articles["paragraphs"]):
        for paragraph in paragraphs:
            for qa in paragraph["qas"]:
                out["title"].append(title)
                out["context"].append(paragraph["context"])
                out["question"].append(qa["question"])
                out["id"].append(qa["id"])
                out["answers"].append({
                    "answer_start": [answer["answer_start"] for answer in qa["answers"]],
                    "text": [answer["text"] for answer in qa["answers"]],
                })
    return out

dataset = dataset.map(process_squad, batched=True, remove_columns=["paragraphs"])
```

I adapted the code from [squad.py](https://github.com/huggingface/datasets/blob/master/datasets/squad/squad.py). The code takes as input a batch of articles (title + paragraphs) and gets all the questions and answers from the JSON structure.

The output is a dataset with `features: ['answers', 'context', 'id', 'question', 'title']`

Let me know if that helps !
