Understanding the role of num_workers in Pytorch's Dataloader
In PyTorch's Dataloader suppose:
I) Batch size=8 and num_workers=8
II) Batch size=1 and num_workers=8
III) Batch size=1 and num_workers=1
with exact same get_item() function.
So,

in case I) will 1 worker be assigned for each batch and in case II) only 1 worker will be used and 7 idle.
Or is it that even in case II) all 8 workers will be used for that loading that single batch
Or is it that 1 worker will be used to load batch for each iteration. I mean say I am on iteration x, and irrespective of batch size batches for future iterations will be pre-loaded as I am using multiple workers?

Finally, will speed of training my CNN be greater in Case II or case III, or will it be same?

Every worker process is always responsible for loading a whole batch, so the batch size and number of workers are not really related.

in case I) will 1 worker be assigned for each batch and in case II) only 1 worker will be used and 7 idle.

All 8 workers will load batches and deliver them whenever required. So as soon as they are done loading their number of batches (defined by prefetch_factor) they just queue up to deliver the data.

Or is it that even in case II) all 8 workers will be used for that loading that single batch

No, there is always just one worker responsible per batch.

Or is it that 1 worker will be used to load batch for each iteration. I mean say I am on iteration x, and irrespective of batch size batches for future iterations will be pre-loaded as I am using multiple workers?

Every batch is loaded by one worker, so if you only load one batch per iteration, only one worker is in taht iteration.
