When training NN, how do data loaders work on large datasets?
For large data sets the limitations of available memory prevents you from loading all the data simultaneously. What is typically done is to provide your data to the network in batches. Batches are simply groupings of your data. The batch size maximum value is limited by available memory. If you use Keras the ImageDataGenerator.flow from directory provides a convenient way to present your data to the network in batches. Documentation is here. There is also a benefit to presenting your data in batches. For example if your data consists of 50,000 samples and it fits into memory you can provide the 50,000 samples to the network and after the data is forward passed do 1 step of back propagation to adjust the weights. A better solution is to partition your data into batches, say 10 batches of 5,000 samples. In this case if the network is enabled to train on batches you can get 10 steps of iteration of the weights versus the single iteration. This enables the network to converge faster at a lower computational cost.