If I have some input, is it better to filter the data before I run my awk action or should I do all the filtering in awk?For example given the following input:$ echo foo\nbar\nbazfoobarbazShould I run:$ echo foo\nbar\nbaz | sed 1q | awk '{ print $0  cats }'foo catsOr:$ echo foo\nbar\nbaz | awk 'NR == 1 { print $0  cats }'foo catsWhy should I run either one?Should I use a different tool?What factors should I be considering?How can I test those factors?
Is it a good idea to filter input before running awk action?
text processing;awk;sed
In this specific case, the second option is the better one.In general, its more efficient to minimise the number of utilities in a pipeline. Its best not to fork (start) unnecessary processes (as in your first example with an unnecessary sed process). On the Internet, its not hard to find examples of complaints about useless uses of cat.With most modern Unix-like systems*, forking is carried out quite efficiently, but it depends on the size of the process being started, e.g, launching perl or python would be much slower than sed or awk.For once-off commands, this doesnt matter too much  but if your pipeline was inside a loop and being run many multiple times, removing unnecessary process from your pipeline could dramatically speed up total execution time.Specific questionsWhy should I run either one?If youre more familiar with the syntax of one over the other, it might bebetter for code readability (and maintainability) to use the tool/languageyour most familiar with.Should I use a different tool?In this specific case, I wouldnt think so. Both awk and sed areappropriate tools for this kind of job.What factors should I be considering?If you have to process multiple files (e.g., in a loop), then speed/efficiency would be important.If youre just processing one large file, every so often, code readabilitymight be more important.How can I test those factors?You can profile different versions by using the time utility, available as shell builtin with Bash  but also as a standalone executable program. E.g. running the two example commands shows that the first example took .012s longer than the second.$ time echo foo\nbar\nbaz | sed 1q | awk '{ print $0  cats }'foo\nbar\nbaz catsreal    0m0.056suser    0m0.000ssys     0m0.045s$ time echo foo\nbar\nbaz | awk 'NR == 1 { print $0  cats }'foo\nbar\nbaz catsreal    0m0.044suser    0m0.000ssys     0m0.031sNote that profiling benchmarks are affected by system load and other limiting factors so youd need to repeat this a large number of times to get a true picture of which version is faster than the other.*With MS Windows, forking is more expensive so minimising the number of processes being started does make a difference when running in environments such as Cygwin.