Split document into multiple files based on pattern
Question: I'm trying to split a large text document of articles into multiple text files
based on a boundary like this:

`9 of 10 DOCUMENTS`

at the beginning of each chunk. Everything after that pattern but before the
next occurence should be written out to a new file. I can do it csplit using:

`csplit -f split-docs/article -b "%02d.txt" -k articles.txt '/[0-9]* of [0-9]*
DOCUMENTS/' {*}`

but I'd also like to be able to do it in python. Thanks for your help.


Answer: You can use
[`itertools.groupby`](http://docs.python.org/2/library/itertools.html#itertools.groupby)
here:

**Demo:**

    
    
    >>> from itertools import groupby
    >>> import re
    >>> r = re.compile(r'\d+ of \d+ DOCUMENTS')
    >>> with open('abc1') as f:
        for k, g in groupby(f, key=lambda x:r.search(x)):
            if not k:
                print list(g)
    ...             
    ['a\n', 'b\n']
    ['c\n', 'd\n']
    ['e\n', 'f\n']
    

where `abc1` contains:

    
    
    >>> !cat abc1
    a
    b
    1 of 3 DOCUMENTS
    c
    d
    2 of 3 DOCUMENTS
    e
    f
    3 of 3 DOCUMENTS
    

