Removing NULL / empty fields just wanted to check with you could this be done better:

awk -F"\t" '{
    for (i = 1; i &lt;= NF; i++) {
        if ($i != "NULL") {
            printf("%s%s", $i, FS);
        }
    }

    printf("\n");
}' file1


The goal is to print only non-NULL fields. For example:

echo "TestRecord001 NULL    NULL    Age 29  NULL    NULL    Name    John" | awk -F"\t" '{
    for (i = 1; i &lt;= NF; i++) {
        if ($i != "NULL") {
            printf("%s%s", $i, FS);
        }
    }

    printf("\n");
}'


will print out: TestRecord001  Age 29  Name    John
The same behaviour can be achieved using sed as follows:

echo -e 'TestRecord001\tNULL\tNULL\tAge\t29\tNULL\tNULL\tName\tJohn' | sed '
s/$/\t/;
s/\(\tNULL\)\+\t/\t/g;
s/^NULL\t//';


Explanation:

sed s/SearchPattern/Replacement/g. Here s indicates that replacement operation is to be done. Strings matching SearchPattern will be replaced by Replacement. g indicates that the operation will have to be performed on every match and not just on the first occurrence in a line.


s/$/\t/ adds a tab to the end of each line. [$ matches the end of a line]
\(\tNULL\)\+\t matches a string of the form \tNULL\tNULL...NULL\t. This is replaced with \t.
After this the only remaining NULL is the one at the beginning of a line (without \t to its left). This is matched by ^NULL\t and replaced with the empty string. [^ matches the beginning of a line]