title: Performance issue with parsing large log files (~5gb) using awk, grep, sed
body: <p>I am currently dealing with log files with sizes approx. 5gb.  I'm quite new to parsing log files and using UNIX bash, so I'll try to be as precise as possible.  While searching through log files, I do the following: provide the request number to look for, then optionally to provide the action as a secondary filter.  A typical command looks like this:</p>
  
  <pre><code>fgrep '2064351200' example.log | fgrep 'action: example'
  </code></pre>
  
  <p>This is fine dealing with smaller files, but with a log file that is 5gb, it's unbearably slow.  I've read online it's great to use sed or awk to improve performance (or possibly even combination of both), but I'm not sure how this is accomplished.  For example, using awk, I have a typical command:</p>
  
  <pre><code>awk '/2064351200/ {print}' example.log
  </code></pre>
  
  <p>Basically my ultimate goal is to be able print/return the records (or line number) that contain the strings (could be up to 4-5, and I've read piping is bad) to match in a log file efficiently.</p>
  
  <p>On a side note, in bash shell, if I want to use awk and do some processing, how is that achieved?  For example:</p>
  
  <pre><code>BEGIN { print "File\tOwner" }
  { print $8, "\t", \
  $3}
  END { print " - DONE -" }
  </code></pre>
  
  <p>That is a pretty simple awk script, and I would assume there's a way to put this into a one liner bash command?  But I'm not sure how the structure is.</p>
  
  <p>Thanks in advance for the help.  Cheers.</p>