❓ Dataset loading script from Hugging Face Hub
Hi ! In the next version of `datasets`, your train and test splits will be correctly separated (changes from #3027) if you create a dataset repository with only your CSV files.

Also it seems that you overwrite the `data_files` and `data_dir` arguments in your code, when you instantiate the AGNewsConfig objects. Those parameters are not necessary since you already know which files you want to load.

You can find an example on how to specify which file the dataset has to download in this [example script](https://huggingface.co/datasets/lhoestq/custom_squad/blob/main/custom_squad.py#L101-L107):
```python
_URLS = {
    "train": "train-v1.1.json",  # you can use a URL or a relative path from the python script to your file in the repository
    "dev": "dev-v1.1.json",
}
```
```python
    def _split_generators(self, dl_manager):
        downloaded_files = dl_manager.download_and_extract(_URLS)

        return [
            datasets.SplitGenerator(name=datasets.Split.TRAIN, gen_kwargs={"filepath": downloaded_files["train"]}),
            datasets.SplitGenerator(name=datasets.Split.VALIDATION, gen_kwargs={"filepath": downloaded_files["dev"]}),
        ]
```
Hi there,

I am trying to add my custom `ag_news` with its own loading script on the Hugging Face datasets hub. In particular, I would like to test the addition of a second configuration to the existing `ag_news` dataset. Once it works in my hub, I plan to make a PR to the original dataset. However, in trying to do so I have encountered certain problems as detailed below.

Issues I have encountered:
- Without a loading script, the train and test files are loaded together into a unique `dataset.Dataset` -> so I wrote a loading script. Also, I need a loading script otherwise I cannot specify multiple configurations
- Once my loading script is working locally, I do not manage to make it work on the hub. In particular, I would like to be able to load the dataset like this
```python
load_dataset("pietrolesci/ag_news", name="my_configuration")
```

Apparently, the `load_dataset` is able to pick up the loading script from the hub and run it. However, it errors because it is unable to find the files. The structure of my hub repo is the following
```
ag_news.py
train.csv
test.csv
```
and the loading script I specify  `data_dir=Path(__file__).parent` and `data_files=DataFilesDict({"train": "train.csv", "test": "test.csv"})`. In the documentation I could not find info regarding loading a dataset from the hub using a loading script present on the hub.

Any suggestion is very much appreciated.

Best,
Pietro

Link to the hub repo: https://huggingface.co/datasets/pietrolesci/ag_news

BONUS: how can I make the data viewer work in this specific case? :)
❓ Dataset loading script from Hugging Face Hub 
 Hi there,

I am trying to add my custom `ag_news` with its own loading script on the Hugging Face datasets hub. In particular, I would like to test the addition of a second configuration to the existing `ag_news` dataset. Once it works in my hub, I plan to make a PR to the original dataset. However, in trying to do so I have encountered certain problems as detailed below.

Issues I have encountered:
- Without a loading script, the train and test files are loaded together into a unique `dataset.Dataset` -> so I wrote a loading script. Also, I need a loading script otherwise I cannot specify multiple configurations
- Once my loading script is working locally, I do not manage to make it work on the hub. In particular, I would like to be able to load the dataset like this
```python
load_dataset("pietrolesci/ag_news", name="my_configuration")
```

Apparently, the `load_dataset` is able to pick up the loading script from the hub and run it. However, it errors because it is unable to find the files. The structure of my hub repo is the following
```
ag_news.py
train.csv
test.csv
```
and the loading script I specify  `data_dir=Path(__file__).parent` and `data_files=DataFilesDict({"train": "train.csv", "test": "test.csv"})`. In the documentation I could not find info regarding loading a dataset from the hub using a loading script present on the hub.

Any suggestion is very much appreciated.

Best,
Pietro

Link to the hub repo: https://huggingface.co/datasets/pietrolesci/ag_news

BONUS: how can I make the data viewer work in this specific case? :) 
 Hi ! In the next version of `datasets`, your train and test splits will be correctly separated (changes from #3027) if you create a dataset repository with only your CSV files.

Also it seems that you overwrite the `data_files` and `data_dir` arguments in your code, when you instantiate the AGNewsConfig objects. Those parameters are not necessary since you already know which files you want to load.

You can find an example on how to specify which file the dataset has to download in this [example script](https://huggingface.co/datasets/lhoestq/custom_squad/blob/main/custom_squad.py#L101-L107):
```python
_URLS = {
    "train": "train-v1.1.json",  # you can use a URL or a relative path from the python script to your file in the repository
    "dev": "dev-v1.1.json",
}
```
```python
    def _split_generators(self, dl_manager):
        downloaded_files = dl_manager.download_and_extract(_URLS)

        return [
            datasets.SplitGenerator(name=datasets.Split.TRAIN, gen_kwargs={"filepath": downloaded_files["train"]}),
            datasets.SplitGenerator(name=datasets.Split.VALIDATION, gen_kwargs={"filepath": downloaded_files["dev"]}),
        ]
```