I have a log file, whose formatting is very consistent and which I control.  It produces pipe-delimited fields of constant length.  Two fields are relevant to the filtering process I wish to create, call them primary and secondary.Using Grep I can first filter on primary.  This will produce an incomplete list of relevant lines.  In this list will show several lines, and these will have one of two distinct values in the secondary field.  There will also be other lines that do not match the primary field, but whose primary field is blank, that match on one or the other of the secondary field values.  All of these rows are relevant.  And I want them in the final output, but I don't know them until I have gone through the initial pass.All entries where the secondary field matches will either have the initial primary field or a blank primary field.  In no case will a secondary field be blank.My strategy is to 1. awk the logfile once, pulling out all rows where the primary field matches (this will be the input for the script).  For each such row, examine the secondary field until both possible matching secondary field values have been found.2. awk the logfile again, pulling out all rows where the primary field matches or the secondary field matches one of two values learned in pass one.I'm an awk newbie  - how can I store the two secondary field values learned in pass 1, and then use them in writing the conditions for pass 2?I've been asked to provide samples so here is a simplified version of the data illustrating the important points.  Primary is field 2 and Secondary is field 3.This is the first pull (input value is 05478900172)2015-03-10 09:13:40,598|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,601|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,601|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,601|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,601|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,617|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,617|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,626|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,626|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:14:16,686|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,694|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,694|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,694|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,695|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,705|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,705|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,714|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,714|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:23,838|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,878|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,878|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,879|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,879|05478900172|4792964 | 43|s|D|S|   02015-03-10 09:14:23,879|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,888|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,888|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:15:01,915|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,917|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,917|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,917|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,917|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,936|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,936|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,945|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,946|05478900172|4792761 | 17|s|D|S|----From this we can see that the secondary field ($3) contains two possible values for this primary field (4792761 or 4792964).We want to have our script pull the following dataset, which contains all records with 05478900172 in $2 and either (4792761 or 4792964) in $3.  I don't know these two values until I've done the initial scan, so I need to pass these values as variables that somehow get shared between the first pass and the second.2015-03-10 09:13:40,598|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,601|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,601|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,601|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,601|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,608|       null|4792761 |---|-|K|-|----2015-03-10 09:13:40,608|       null|4792761 |---|-|K|-|----2015-03-10 09:13:40,617|       null|4792761 |---|r|D|S|----2015-03-10 09:13:40,617|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,617|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,626|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:13:40,626|05478900172|4792761 | 15|s|D|S|----2015-03-10 09:14:16,686|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,694|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,694|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,694|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,695|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,696|       null|4792964 |---|-|K|-|----2015-03-10 09:14:16,696|       null|4792964 |---|-|K|-|----2015-03-10 09:14:16,704|       null|4792964 |---|r|D|S|----2015-03-10 09:14:16,705|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,705|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,714|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,714|05478900172|4792964 | 41|s|D|S|----2015-03-10 09:14:16,760|       null|4792964 |---|r|D|S|----2015-03-10 09:14:16,760|       null|4792964 |---|r|D|S|----2015-03-10 09:14:23,817|       null|4792964 | 42|-|D|S|----2015-03-10 09:14:23,817|       null|4792964 | 42|-|D|S|----2015-03-10 09:14:23,817|       null|4792964 | 42|-|D|S|70572015-03-10 09:14:23,817|       null|4792964 | 42|-|D|S|----2015-03-10 09:14:23,818|       null|4792964 | 42|-|D|S|----2015-03-10 09:14:23,818|       null|4792964 | 42|-|D|S|----2015-03-10 09:14:23,838|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,876|       null|4792964 |---|-|K|-|----2015-03-10 09:14:23,876|       null|4792964 |---|-|K|-|----2015-03-10 09:14:23,878|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,878|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,878|       null|4792964 |---|r|D|S|----2015-03-10 09:14:23,879|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,879|       null|4792964 |---|r|D|S|----2015-03-10 09:14:23,879|05478900172|4792964 | 43|s|D|S|   02015-03-10 09:14:23,879|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,888|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:14:23,888|05478900172|4792964 | 43|s|D|S|----2015-03-10 09:15:01,915|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,917|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,917|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,917|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,917|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,925|       null|4792761 |---|-|K|-|----2015-03-10 09:15:01,925|       null|4792761 |---|-|K|-|----2015-03-10 09:15:01,936|       null|4792761 |---|r|D|S|----2015-03-10 09:15:01,936|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,936|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,945|05478900172|4792761 | 17|s|D|S|----2015-03-10 09:15:01,946|05478900172|4792761 | 17|s|D|S|----
How to Save variables in a script that can be shared between two runs of awk against the same input file in the script?
shell script;awk