Given list of image filenames for eachset,Split large dataset to train/valid/test directories?
I am trying to split a large dataset into train/valid/test sets from Food101 dataset  for image classification 

and the structure of a dataset is like this and has all images in one folder

'',
'Structure:',
'----------',
'pec/',
'    images/',
'        &lt;class_name&gt;/',
'            &lt;image_id&gt;.jpg',
'    meta/',
'        classes.txt',
'        labels.txt',
'        test.json',
'        test.txt',
'        train.json',
'        train.txt',
'',
'All images can be found in the "images" folder and are organized per class. All',
'image ids are unique and correspond to the foodspotting.com review ids. 
'',
'The test/train splitting used in the experiment of our paper can be found in',
'the "meta" directory.', (edited) ```



I want to divide images dataset to train/valid/test  with the list of filenames given in train.txt and test.txt, which author used 



the Shape of train,valid, test lists:  (101, 600) , (101, 150) , 25250

In colab ,i run following code


for x in range(train.shape[0]):
    for y in range(train.shape[1]):

     temp = train[x,y] + ".jpg"

     foldername = temp.split('/')[0]

     !mv /content/food-101/images/$temp /content/food101/train/$foldername/


Individually moving images by running nested loop by taking filenames in lists, is  taking forever time to create folders as there are 100100 images total so,

I have a list of filenames for train/valid and test set but how to make them into folders so that we can feed it to image classifier in pytorch image folder format(i mean train / valid / test set are three different folders and each folder has subfolders of each class)

Please Do tell if anyone knows how to do this please, and please I really need your help here,  THanks:smile:

It seems i have been going all wrong about the solution, i need not move the images all i need to change is path to images in the required format through os module 

Below is the code for doing it. Say you have list of filenames in valid list

#for valid set 

v = valid.reshape(15150,)

or_fpath = '/content/food-101/images/' #path of original folder
cp_fpath = '/content/food101/valid/'   #path of destination folder

for y in tqdm(v):

 foldername = y.split('/')[0]

 img = y.split('/')[1] +'.jpg'

 ip_path = or_fpath+foldername
 op_path = cp_fpath+foldername

 if not os.path.exists(op_path):
   os.mkdir(op_path)

 os.rename(os.path.join(ip_path, img), os.path.join(op_path, img))





Thanks!

note: if you have even better answer please share Thanks
