I use the tail, head and grep commands to search log files. Most of the time the combination of these 3 commands, in addition to using pipe, gets the job done. However, I have this one log that many devices report to literally every few seconds. So this log is very large. But the pattern of the reporting is the same:Oct 10 11:58:50 Received Packet from [xxx.xx.xxx.xx:xxxx]: 0xD 0xD 0xD Oct 10 11:58:50 Unit ID: 1111In the above example, it shows that UDP packet was sent to the socket server for a specific unit id.Now sometimes I want to view the packet information for this unit within a specific time range by quering the log. Oct 10 11:58:50 Received Packet from [xxx.xx.xxx.xx:xxxx]: 0xD 0xD 0xD Oct 10 11:58:50 Unit ID: 1111... // A bunch of other units reporting including unit id 1111Oct 10 23:58:50 Received Packet from [xxx.xx.xxx.xx:xxxx]: 0x28 0x28 0x28 Oct 10 23:58:50 Unit ID: 1111So in the example above, I would like to display log output only for Unit ID: 1111 within the time range of 11:58 and 23:58. So the possible results can look like this:Oct 10 11:58:50 Received Packet from [xxx.xx.xxx.xx:xxxx]: 0xD 0xD 0xD Oct 10 11:58:50 Unit ID: 1111Oct 10 12:55:11 Received Packet from [xxx.xx.xxx.xx:xxxx]: 0x28 0xD 0x28 Oct 10 12:55:11 Unit ID: 1111Oct 10 15:33:50 Received Packet from [xxx.xx.xxx.xx:xxxx]: 0x33 0xD 0x11 Oct 10 15:33:50 Unit ID: 1111Oct 10 23:58:50 Received Packet from [xxx.xx.xxx.xx:xxxx]: 0x28 0x28 0x28 Oct 10 23:58:50 Unit ID: 1111Notice the results only display Unit ID: 1111 information and not the other units.Now the problem with using something like this:tail -n 10000 | grep -B20 -A20 Oct 10 23:58:50 Unit ID: 1111 is that will display a lot of stuff, not just the stuff that I need.
Search and Filter Text on large log files
grep;tail;text;filter;head
awk '$3 >= 11:58 && $3 <= 23:58 && /Unit ID: 1111/{print l\n$0};{l=$0}'