Comparing a list of known animals to a master list of animals This script works for what I need it to do, but I'm not all that proficient with Python yet, so I was wondering if any of the more seasoned python veterans would go about it differently.

import os
import string
import re

'''This script will take in an input list of strictly stellar IDs, and compare those IDs to a master file 
with IDs AND photometric data. It will then write to a file, the lines from the master file that correspond to the known IDs 
in the IDs list.'''

def pullIDs(file_input):
    '''Pulls Mon-IDs from input file.'''
    arrayID = []
    with open(file_input,'rU') as user_file:
        for line in user_file:
            arrayID.append(re.findall('Mon\-\d{6}',line))
    return arrayID

def pullLines(file_input):
    '''Creates an array component for each
    line in the input file.'''
    arrayLines = []
    with open(file_input,'rU') as info:
        for item in info:
            item.split
            arrayLines.append(item)
    return arrayLines

def getHeader(file_input):
    '''Assuming the file-header begins with a '#',
    this will pull the header line.'''
    with open(file_input,'rU') as title:
        for line in title:
            if line.startswith('#'):
                header = line
    return header

known_stars = raw_input("Enter your ID list: ")
master_list = raw_input("Enter your master list: ")
results_list = raw_input("Name your output file: ")

with open(results_list,"w+") as output:
    output.write(getHeader(master_list))
    for item in pullIDs(known_stars):
        for items in pullLines(master_list):
            if items[:10] in item:
                output.write(items)

output_fileSIZE = os.stat(results_list).st_size
if output_fileSIZE &gt; 0:
    print "\nYour output file has been written to.\n"
else:
    print "\nThere was an error writing to your file.\n"


This essentially takes a list of known animals, say, cat and dog, and compares that list to a master list of animals and their respective information.

For example:


cat female 2 gray playful
dog male 9 black aggressive
giraffe male 12 tall hungry
elephant male 33 gray lonely
etc.



By creating two arrays for both lists, it searches for an animal that exists in both files and, once found, prints all of the more in-depth results from the 'master list' to a file.

So my file would look like this:


cat female 2 gray playful
dog male 9 black aggressive



because I only had a 'known list' of cats and dogs. 

I'm just looking for tips on how to make this script better.  It also prints a header for the column information.
This is the sort of thing that Python is really good at.  Your code seems like it functions, but it will be very inefficient for large data sets. For each input value you have to check every line in the the master list -- and the way you have it set up here you're going to read the master list off disk for each input value as well, so it will be really slow.

The first step obvious fix is to load the master list into memory and keep it around for the duration of a session.  This is a fairly simple change to your existing code.  First, change your pull_lines so it returns a dictionary instead of a list:

def pullLines(file_input):
    '''returns a dictionary of the entries in &lt;file_input&gt; keyed by the first item in each line'''
    master_dict = {}
    with open(file_input,'rU') as info:
        for eachline in info.readlines():
            tokens = eachline.split()
            master_dict[tokens[0]] = tokens[1:]


Then, you move pull lines out of the inner loop:

lookup = pull_lines(path_to_file)
with open(results_list,"w+") as output:
    output.write(getHeader(master_list))
    for item in pullIDs(known_stars):
        if item in lookup:
            output.write(lookup[item])


This has a couple of advantages

1) not hitting the disk for every query will be much faster

2) the dictionary lookup faction is much faster than straight for loop comparison, since the dictionary uses hashed values instead of more expensive string compares.

As an aside, you should look into readlines, which the usual pythonic way of reading text files with info on lines.

For the longer term this is a great application for Python's built in database functionality (with the sqllite module). You could convert your existing text files into a sqllite database and then your lookups can be much more flexible and precise ('find spectrum and power for stars with id > X and &lt; Y' sort of thing).  You probably also want to set this file up so you could call it from the command line, which would involve using the argparse module to grab command line arguments and out them into known_stars, master_list and output_file