Removing a row from CSV with python if data wasn't recorded in a column
Question: I'm trying to import a batch of CSV's into PostgreSQL and constantly run into
an issue with missing data:

> psycopg2.DataError: missing data for column "column_name" CONTEXT:  
>  COPY table_name, line _where ever in the CSV that data wasn't  
>  recorded, and here are data values up to the missing column_.

There is no way to get the complete set of data written to the row at times,
and I have to deal with the files as is. I am trying to figure a way to remove
the row if data wasn't recorded into any column. Here's what I have:

    
    
    file_list = glob.glob(path)
    
    for f in file_list:
        filename = os.path.basename(f) #get the file name
        arc_csv = arc_path + filename #path for revised copy of CSV
    
        with open(f, 'r') as inp, open(arc_csv, 'wb') as out:
            writer = csv.writer(out)
            for line in csv.reader(inp):
                if "" not in line: #if the row doesn't have any empty fields
                    writer.writerow(line)
    
        cursor.execute("COPY table_name FROM %s WITH CSV HEADER DELIMITER ','",(arc_csv,))
    


Answer: Unfortunately, you _cannot parameterize table or column names_. Use string
formatting, but make sure to validate/escape the value properly:

    
    
    cursor.execute("COPY table_name FROM {column_name} WITH CSV HEADER DELIMITER ','".format(column_name=arc_csv))
    

