Search and return lines in a file with matching keyword in python
Question: I have used regex to isolate a particular keyword in a line taken from a file.
I want to search the entire file and return groups of lines which have the
same keyword.

I am a little confused about this, and I was wondering if there is a direct
regex way to do this in Python ?

e.g. - >

My file may look like this

    
    
    1  0001    1   UG  science,ee;YEAR=onefour;standard->2;district->9
    2  0002    1   UG  science,cs;YEAR=onefive;standard->1;district->9
    3  0012    2   UG  science,eng;YEAR=onefour;standard->3;district->4
    4  0021    2   UG  science,ee;YEAR=onetwo;standard->2;district->9
    5  0056    4   UG  science,cs;YEAR=onefive;standard->1;district->8
    6  0145    3   UG  science,eng;YEAR=onetwo;standard->4;district->2
    

I used regex to extract

    
    
    "YEAR=****" 
    

and want to group the lines according to the value of

    
    
    "****"
    

The output should look like this -

    
    
    1  0001    1   UG  science,ee;YEAR=onefour;standard->2;district->9
    3  0012    2   UG  science,eng;YEAR=onefour;standard->3;district->4
    
    2  0002    1   UG  science,cs;YEAR=onefive;standard->1;district->9
    5  0056    4   UG  science,cs;YEAR=onefive;standard->1;district->8
    
    4  0021    2   UG  science,ee;YEAR=onetwo;standard->2;district->9
    6  0145    3   UG  science,eng;YEAR=onetwo;standard->4;district->2
    

I believe I can do it the long way of opening file, storing in dictionaries
and matching, etc, etc. But would like to know if there is short concise way
of doing this.

as requested - a bit of code I tried to write and run -

    
    
    #!/usr/bin/python
    
    import re
    
    ##open file and read each line of file
    
    dfile = open("datafile.txt","r")
    
    ##regex to find YEAR in entry and return YEAR
    
    regex_unique = re.compile(r'(?<=\bYEAR=)[^;]+')
    
    list_Name =[]
    
    for line in dfile:
        match1 = re.search(regex_unique,line)
        if match1:
            if match1.group(0) not in list_Name:
            list_Name.append(match1.group(0))
    
    
    ## print (list_Name)
    
    for item in list_Name:
    for line in dfile:
        match2 = re.search(item,line)
        if match2:
            print (match2)
    

the last bit does not seem to work - I am assuming if I give

    
    
    item
    

to

    
    
    re.search
    

it should search for that word in the entire file - Now I think I might have
to add some wildcard entries before and after the actual word to make it work.


Answer: I think I'm right in saying that regex only deals with matches on lines, not
with how to aggregate matches -- so you'll need to do that yourself. You can
keep things simple by writing your own utility function and keeping it
separate from you application code.

Grouping operations in general must pass over all of the items to assemble the
groups. Your problem can't be solved without making a pass over all of the
data to collect the groups, and then another pass to output the groups.

A dictionary of lists is the natural datastructure to collect a each line by
by a key (as you note). Doing this yourself set is a little kludgey, since you
frequently need to test whether a key exists to know whether you should add to
an existing list or create a new one. Fortunately, python provides
defaultdict, that lets you:

    
    
    from collections import defaultdict
    >>> d = defaultdict(list)
    >>> d[key].append(line)
    

So, you can do the following:

    
    
    def groupLinesByMatch(filename,regex):
        import re
        from collections import defaultdict
    
        regex = re.compile(regex)
        result = defaultdict(list)
    
        for line in open(filename).readlines():
            matches = regex.match(line)
            if matches:    
                result[matches.group(1)].append( line )
    
        return result.values()
    
    
    for lines in groupLinesByMatch(filename, regex):
        for line in lines:
            print line,
        print
    

