Python 3.4 : match from csv and return new csv with matched values
Question: # QUESTION

How can I scan the reader csv for any items in the reader2 csv and return a
new csv with the matched information.

# Reader2 csv format

    
    
    66740,1800,1001463,1467373,896159
    

# reader csv format

    
    
    1001385|NORTHWEST PIPE CO|10-Q|2015-05-06|edgar/data/1001385/0001193125-15-174814.txt
    1001426|PERICOM SEMICONDUCTOR CORP|10-Q|2015-05-05|edgar/data/1001426/0001145443-15-000628.txt
    1001463|Acacia Diversified Holdings, Inc.|10-K|2015-05-20|edgar/data/1001463/0001185185-15-001386.txt
    1001463|Acacia Diversified Holdings, Inc.|10-K|2015-05-20|edgar/data/1001463/0001185185-15-001394.txt
    1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001388.txt
    1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001390.txt
    1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001392.txt
    1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001396.txt
    

# Current Code

    
    
    with open('newCIK.csv') as reader2:
        reader2 = csv.reader(reader2)
    
        with open('search.file') as f_in, open('SP500_10K.csv', 'w') as f_out:
            reader = csv.reader(f_in, delimiter='|')
            writer = csv.writer(f_out, delimiter='|')
    
            for line in reader:
                for cik in reader2:
                    if cik in line:
                        writer.writerow(line)
    


Answer: You are trying to treat a file object as a list, looping over it more than
once. That won't work without doing extra work. Moreover, you are not looping
over the columns of the one row; you are trying to test if the _whole row_ is
in the other CSV file rows. You'd want to test each value, and then only
against the last column of the rows in the `search.file` CSV data.

File objects have a _file position_ ; as you read from the file the position
moves from start to end. Once at the end _it won't move to the start again
automatically_.

You could seek the file object to the start again:

    
    
    with open('newCIK.csv') as reader2_file:
        reader2 = csv.reader(reader2_file)
    
        with open('search.file') as f_in, open('SP500_10K.csv', 'w') as f_out:
            reader = csv.reader(f_in, delimiter='|')
            writer = csv.writer(f_out, delimiter='|')
    
            for line in reader:
                reader2_file.seek(0)  # rewind to the start
                for cik in reader2:
                    if cik in line:
                        writer.writerow(line)
    

However, reading a file over and over is _slow_. You'd be better of reading
the whole thing into memory at the start. And the above doesn't address the
other problem, namely that you are testing each row, and not each column, from
`newCIK.csv`.

Read the _one_ row into memory, then loop over that:

    
    
    with open('newCIK.csv', newline='') as reader2:
        reader2 = csv.reader(reader2)
        cik_values = next(reader2)  # first row
    
    with open('search.file', newline='') as f_in, open('SP500_10K.csv', 'w', newline='') as f_out:
        reader = csv.reader(f_in, delimiter='|')
        writer = csv.writer(f_out, delimiter='|')
    
        for line in reader:
            for cik in cik_values:
                if cik in line[-1]:  # test only the last column
                    writer.writerow(line)
    

Note that I added in `newline=''` arguments to the `open()` calls; the `csv`
module needs more control over newlines; not doing so could cause problems on
Windows and when handling values containing newlines.

Demo:

    
    
    >>> from io import StringIO
    >>> import csv, sys
    >>> newcik = '''\
    ... 66740,1800,1001463,1467373,896159
    ... '''
    >>> search_file = '''\
    ... 1001385|NORTHWEST PIPE CO|10-Q|2015-05-06|edgar/data/1001385/0001193125-15-174814.txt
    ... 1001426|PERICOM SEMICONDUCTOR CORP|10-Q|2015-05-05|edgar/data/1001426/0001145443-15-000628.txt
    ... 1001463|Acacia Diversified Holdings, Inc.|10-K|2015-05-20|edgar/data/1001463/0001185185-15-001386.txt
    ... 1001463|Acacia Diversified Holdings, Inc.|10-K|2015-05-20|edgar/data/1001463/0001185185-15-001394.txt
    ... 1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001388.txt
    ... 1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001390.txt
    ... 1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001392.txt
    ... 1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001396.txt
    ... '''
    >>> with StringIO(newcik) as reader2:
    ...     reader2 = csv.reader(reader2)
    ...     cik_values = next(reader2)  # first row
    ... 
    >>> with StringIO(search_file) as f_in:
    ...     reader = csv.reader(f_in, delimiter='|')
    ...     writer = csv.writer(sys.stdout, delimiter='|')
    ...     for line in reader:
    ...         for cik in cik_values:
    ...             if cik in line[-1]:  # test only the last column
    ...                 writer.writerow(line)
    ... 
    1001463|Acacia Diversified Holdings, Inc.|10-K|2015-05-20|edgar/data/1001463/0001185185-15-001386.txt
    103
    1001463|Acacia Diversified Holdings, Inc.|10-K|2015-05-20|edgar/data/1001463/0001185185-15-001394.txt
    103
    1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001388.txt
    103
    1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001390.txt
    103
    1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001392.txt
    103
    1001463|Acacia Diversified Holdings, Inc.|10-Q|2015-05-20|edgar/data/1001463/0001185185-15-001396.txt
    103
    

The `103` numbers are the number of bytes written in each `writer.writerow()`
call, echoed by the REPL.

