I have 200 files in a folder like: test_1_cfg.dat, test_2_cfg.dat,.... and so on. I need to include in a bash script the first 40 files as input and run some process, another script to read the next 40 files and run another process. So I was thinking of a way to have a list of the names of the files and then just split that list but I'm not sure how to do it in bash.Any idea?
Bash split a list of files
bash;shell script;split
Method #1 - Using head & tailYou can use the command head to pull out the first 40 files from a file listing like so:$ head -40 input_files | xargs ...To get the next 40:$ tail -n +41 input_file  | head -40 | xargs ......$ tail -n +161 input_file | head -40 | xargs ...You can keep walking down the list, 40 at a time using this same technique.Method #2 - Using xargsIf you happen to have all your filenames in a variable, you can use xargs like so to break up the list into chunks of X number of elements.ExamplePretend my files are called 1-200. So I load them up into a variable like so:$ files=$(seq 200)You can see the first couple of items in this variable:$ echo $files  | head -c 201 2 3 4 5 6 7 8 9 10Now we use xargs to divide it up:$ xargs -n 40 <<<$files1 2 3 4 5 6 7 8 9 10 ...41 42 43 44 45 46 47 ...81 82 83 84 85 86 87 ...121 122 123 124 125 ...141 142 143 144 145 ...161 162 163 164 165 ...181 182 183 184 185 ...You could then pass the above command to another xargs which would then run your program:$ xargs -n 40 <<<$files | xargs ...If the contents of the list of files isn't easily accessible from a variable you can give xargs a list via a file instead:$ xargs -n 40 <input_file1 2 3 4 5 6 7 8 9 10 ...41 42 43 44 45 46 47 ...81 82 83 84 85 86 87 ...121 122 123 124 125 ...141 142 143 144 145 ...161 162 163 164 165 ...181 182 183 184 185 ...Method #3 - Bash arraysSay you had your filenames in a Bash array. Again I'm using a sequence of number 1-200 to represent my filenames.$ foo=( $(seq 200) )You can see the contents of the array like so:$ echo ${foo[@]}1 2 3 4 5 ....Now to get the 1st 40:$ echo ${foo[@]:0:40}The 2nd 40, etc:$ echo ${foo[@]:40:40}...$ echo ${foo[@]:160:40}