I have a file here where I want to tag each line with FILEB and FILEC as references. If column 1 of fileb or filec  is in column 2 of filea, get the column 2 of fileb or filec (if column 1 file c not equal to 63), else print 'others' (see OUTPUT1). The final output would be the sum of column 4 (group by column 2 & 5 of OUTPUT1) and count per column 5 (OUTPUT 1).FILEA 63,234111,000,2 63,234111,111,3 56,456711,000,2 63,678999,111,1FILEB 234,XXX 456,ZZZFILEC 4567,YYY 234,GGGOUTPUT1 63,234111,000,2,XXX 63,234111,111,3,XXX 56,456711,000,2,YYY 63,678999,111,1,OTHERSFINAL C1, C2,     C3,  SUM of C4, XXX, ZZZ, YYY, GGG, OTHERS 63, 234111, 000, 5,         2,   0,   0,   0,   0 56, 456111, 000, 2,         0,   0,   1,   0,   0 63, 678999, 111, 1,         0,   0,   0,   0,   1I have a scripts here but too slow to process very large file.##tagging (ref fileb)awk -F~ 'NR==FNR {a[$1+0]=$2;next} $1+0==63 {print $0~a[$2+0]}' fileb filea > OUTPUT##tagging (ref filea)awk -F~ 'NR==FNR {a[$1+0]=$2;next} $1+0!~63 {print $0,a[$2+0]}' filec OUTPUT > OUTPUT##tagging othersawk -F~ '{if ($5 == ) print $0,OTHERS}' > OUTPUT##sum and countawk 'BEGIN { FS=OFS=SUBSEP=~}{arr[$2,$5]+=$4 }{arr2[$2,$5]++}END {for (i in arr) print i,arr[i],arr2[i]}'
awk to process very large file
awk;performance