I have scenario where my three huge files Test.txt , Test1.txt and Test2.txt has following details.H|||||||||||||||||||||||D||||||||||||||||||||||||D|||||||||||||||||||||||H|||||||||||||||||||||D||||||||||||||||||||||||D||||||||||||||||||||||||T||||||||||||||||||||||||I have to delete all except D lines.It should look like below in all my three files.(more than 10 GB)D||||||||||||||||||||||||D|||||||||||||||||||||||D||||||||||||||||||||||||D||||||||||||||||||||||||So after retaining only D's lines in Test.txt, Test2.txt and Test3.txt, I have to merge those into new file.I have done the above operation using sed. sed '/^\('D'\)|/!d' $Filename.txt >>  $NewFilename.txtBut because of huge file its taking very long time.Can we do this operation using any other command in efficient way?
Remove all lines except D
shell script
cat Test.txt Test2.txt Test3.txt | LC_ALL=C grep '^D' > newfile.txtOr:for file in Test.txt Test2.txt Test3.txt; do  LC_ALL=C grep '^D' < $filedone > newfile.txtOr if your grep like GNU grep supports the -h option (to avoid printing file names):LC_ALL=C grep -h '^D' Test.txt Test2.txt Test3.txt > newfile.txtBy using LC_ALL=C we avoid grep trying to parse UTF-8 data. By using ^D, grep will only look at the first character of each line. grep, especially GNU grep is generally a lot faster than sed.