The actual log file will be like this$ cat file1.logtime=2014-07-23 23:56:28 GMT, user=[unknown], db=[unknown], host= pid=28254 LOG: time=2014-07-23 23:56:28 GMT, user=portalman, db=ss, host=172.18.183.45 pid=28254 LOG:  connection authorized: user=portalman database=sstime=2014-07-23 23:57:28 GMT, user=[unknown], db=[unknown], host= pid=28269 LOG:  connection received: host=172.18.183.45 port=14493time=2014-07-23 23:57:28 GMT, user=portalman, db=ss, host=172.18.183.45 pid=28269 LOG:  connection authorized: user=portalman database=sstime=2014-07-23 23:57:28 GMT, user=[unknown], db=[unknown], host= pid=28270 LOG:  connection received: host=172.18.183.45 port=14494time=2014-07-23 23:57:28 GMT, user=portalman, db=ss, host=172.18.183.45 pid=28270 LOG:  connection authorized: user=portalman database=sstime=2014-07-23 23:57:58 GMT, user=[unknown], db=[unknown], host= pid=28273 LOG:  connection received: host=172.18.183.45 port=14495                                            column actice does not exist at character 17...The total lines will be around 20,000. In this log file I need to fetch the unique errors, which I got through this command:$ find file1.log | xargs grep -e ERROR -e FATAL | cut -d : -f4,5 |sort |uniqFollowing is the result set and it not constant, it variessyntax error at or near ? at character 1syntax error at or near [ at character 1syntax error at or near desc at character 1syntax error at or near describtion at character 1For getting the process id (pid) of each error I used:$ find file1.log | xargs grep 'syntax error at or near ? at character 1' | cut -d   -f8 | tail -1pid=25997Now I got the process id,then I need to fetch the entire information of that process id$ find file1.log  | xargs grep pid=25997time=2014-07-23 23:10:02 GMT, user=[unknown], db=[unknown], host= pid=25997 LOG:  connection received: host=[local]time=2014-07-23 23:10:02 GMT, user=dbman, db=ss, host=[local] pid=25997 LOG:  connection authorized: user=dbman database=sstime=2014-07-23 23:10:02 GMT, user=dbman, db=ss, host=[local] pid=25997 ERROR:  syntax error at or near ? at character 1time=2014-07-23 23:10:02 GMT, user=dbman, db=ss, host=[local] pid=25997 STATEMENT:  ?column?   How to put all the 3 find commands in a shell script & do the process automatically? The errors & number of lines varies. Nothing is constant.
Shell scripts for the database logs
shell script;text processing