I have a very large file (~700M rows) and I would like to reduce the size by grouping mostly matching rows. Specifically, the file is sorted by fields 1 and 2 and I would like to group rows where field 2 contains consecutive numbers but all other fields match. If there is a gap in field 2 or if any other fields do not match the previous row then I would like to start a new interval. Ideally, I would like the output to return the interval range for the grouped rows and would prefer a solution that works in bash with awk and/or sed. I'm open to other solutions as well as long as they don't require re-sorting or other operations that might crash with such a long file.

The input file looks something like this.

```
NW_005179401.1  100 1   0   0   0   0   0   0   0   0
NW_005179401.1  101 1   0   0   0   0   0   0   0   0
NW_005179401.1  102 1   0   0   0   0   0   0   0   0
NW_005179401.1  103 1   0   0   0   0   0   1   0   0
NW_005179401.1  104 1   0   0   0   0   0   1   0   0
NW_005179401.1  105 1   0   0   0   0   0   1   0   0
NW_005179401.1  106 1   0   0   0   0   0   1   0   0
NW_005179401.1  108 1   0   0   0   0   0   1   0   0
NW_005179401.1  109 1   0   0   0   0   0   1   0   0
NW_005179401.1  110 1   0   0   0   0   0   1   0   0
NW_005179401.1  111 1   0   0   0   0   0   1   0   0
NW_005179401.1  112 1   0   0   0   0   0   1   0   0
NW_005179401.1  992 0   0   1   1   0   0   0   0   2
NW_005179401.1  993 0   0   1   1   0   0   0   0   2
NW_005179401.1  994 0   0   1   1   0   0   0   0   2
NW_005179401.1  995 0   0   1   1   0   0   0   0   2
NW_005179401.1  996 0   0   1   1   0   0   0   0   0
NW_005179401.1  997 0   0   1   1   0   0   0   0   0
NW_005179401.1  998 0   0   1   1   0   0   0   0   0
NW_005179401.1  999 0   0   1   1   0   0   0   0   0

```

In reality the file has more fields but all contain integers like fields 3 and beyond in the example. The ideal output will look like this, with first and last values from consecutive field 2 interval printed in output fields 2 and 3.

```
NW_005179401.1  100 102 1   0   0   0   0   0   0   0   0
NW_005179401.1  103 106 1   0   0   0   0   0   1   0   0
NW_005179401.1  108 112 1   0   0   0   0   0   1   0   0
NW_005179401.1  992 995 0   0   1   1   0   0   0   0   2
NW_005179401.1  996 999 0   0   1   1   0   0   0   0   0

```

I found solutions group consecutive rows with matches in specific fields, but none that also look for consecutive integers in one field and not one that can return the range. One thought was using uniq with the -c flag while skipping the first 2 fields, then adding the counts to the value in field 2, but given the additional condition of requiring consecutive numbers in field 2 I'm not too sure where to start with this one. Thanks in advance.

EDIT: I apologize for not originally adding my attempted code but my pipeline used the bioinformatics program bedtools and it kept getting killed for lack of memory, which wasn't something I expected to be troubleshot due to lack of pre-programmed functionality. I am an awk novice and didn't know where to start for an alternative pipeline for reformatting this type of file.