Look ahead without itertools
Question: I am looking for a way to look at the next line in a text file when the first
characters are the letters are only A,G,C,U or N. I created a dict. of all
possibilities in which I can look. I have tried itertools, but to no avail and
I have heard that the itertools would keep everything in memory which would be
most unproductive since my files are rather large(>10GB sometimes). I would
really appreciate help, I have wandered here for days looking for an answer. I
was thinking or trying regex, but I do not know how to. I really want to find
the most productive way for big files. Here is my (pitiful) attempt.

I have taken part of an answer found on : [Python for-loop look-
ahead](http://stackoverflow.com/questions/4197805/python-for-loop-look-ahead)

    
    
    f2 = open(path to file)
    from itertools import tee
    from itertools import permutations
    
    def pairwise(iter):
       "s -> (s0,s1), (s1,s2), (s2, s3), ..."
        a, b = tee(iter)
        next(b, None)
        return zip(a, b)
    
    p = permutations(['A','G','U','C','N'])
    for per in p:
        per = ''.join(per)
        dic={'a':[]}
        dic['a'].append(per)
    
    for line, next_line in pairwise(f2):
    
    
        if line in dic['a']:
    
            letter= next_line.split()
            unilist.append('%s' %next_line)
            print (unilist)
    

It appears the problem lies in the: for line, next_line in pairwise(f2) I
would be truly grateful for every tip and advice.

Edit: I meant the characters in the line and not the ones in the next_line.


Answer: You have several problems with your code:

  * You discard `dic` each permutation loop iteration and re-create it from scratch. Build it once:
    
        dic={'a':[''.join(per) for per in permutations('AGUCN')]}
    

but for fast membership tests (`if something in sequence`), use a `set`
instead:

    
        dic={'a': set(''.join(per) for per in permutations('AGUCN'))}
    

Note that it is not clear why you need a dictionary with one key; a simple
variable would do fine here.

  * You read a file but don't strip the newlines from the lines. With a newline, your `if line in dic['a']` test will never return True because the code that generates the 

Just treat the file as an iterable, call `next()` on it to get the next line:

    
    
    from itertools import permutations
    
    patterns = set(''.join(per) for per in permutations('AGUCN'))
    unilist = []
    
    for line in f2:
        if line.strip() in patterns:
            unilist.append(next(f2).strip())
    

or even:

    
    
    from itertools import permutations
    
    patterns = set(''.join(per) for per in permutations('AGUCN'))
    unilist = [next(f2).strip() for line in f2 if line.strip() in patterns]
    

You are not really looking ahead. You are looking behind; if the previous line
matched a condition, the next line is appended.

