Pytorch: How to load csv files in several folder
My data like:
folder 1
  part0001.csv
  part0002.csv
  ...
  part0199.csv
folder 2
  part0001.csv
  part0002.csv
  ...
  part0199.csv
folder 3
  part0001.csv
  part0002.csv
  ...
  part0199.csv

Update:
Each .csv file is about 100Mb. Both features and label are in the same .csv file. Each .csv file is like as follows.
  feat1 feat2 label
1 1     3     0
2 3     4     1
3 2     5     0
...

I want to load the samples in the .csv file in batch.

You have to build a dataset that loads them. (docs: https://pytorch.org/docs/stable/data.html#torch.utils.data.Dataset)
Example:
import torch
from torch.ults.data import Dataset
import glob2
import pandas as pd

class CustomDataset(Dataset):

    def __init__(self, root)
        self.root = root

        # make a list containing the path to all your csv files
        self.paths =  glob2.glob('src/**/*.csv)
    
    def __len__(self):
        return len(self.paths)

    def __getittem__(idx):

        data = pd.from_csv(self.paths[idx])
        x = data['features']
        y = data['labels']

        return x, y


That is the basic, you can modify it to sample random examples from each csv file or pre-process the data before training.
Edit
If you are just intereted in one line from the csv, there are three things you can do.

Pre-process your data and save it as one big .csv file and load it all in memory before training. That will save you from the trouble of reloading heavy files.
(If the previous is not possible because of the final file would nof fit in memory) Pre-process your data and save it  .csv file per datapoint. That will still require your dataloader to read from disc, but at least you will be loading lighter files this time.
(If preprocessing your data is not an option) Keep as much as you can in memory to avoid reloading files.

There aren't many secrets to implementing the first two solutions. The code for solution 3 should look something like this:
import torch
from torch.ults.data import Dataset
import glob2
import pandas as pd

class CustomDataset(Dataset):

    def __init__(self, root)
        self.root = root

        # make a list containing the path to all your csv files
        self.paths =  glob2.glob('src/**/*.csv)
        
        # dict to keep load data in memory:
        self.cache = {}

    
    def __len__(self):
        return len(self.paths)

    def __getittem__(idx):
        &quot;&quot;&quot;This getittem will load data and save them in memory during training.&quot;&quot;&quot;
        data = cache.get(idx, None)
        
        if data is None:

            data = pd.from_csv(self.paths[idx])
            
            try:
                # cache data into memory
                self.cache{idx: data}
            except OSError:

                # we may be using too much memory
                del self.cache[list(self.cache.keys())[0]]

        rnd_idx = np.random.randint(len(data))
        x = data['features'][rdn_idx]
        y = data['labels'][rdn_idx]
 
        return x, y


