Sqlite3 / python - Export from sqlite to csv text file does not exceed 20k
Question: I am attempting to export a sqlite table to a text file and I found some great
help at this site. It works great for smaller outputs, but once I reach around
20k it appears to limit the output.

# first attempt was:

Mark Bells UniCodeWriter as found in [It is possible export table sqlite3
table to csv or similiar?](http://stackoverflow.com/questions/4264379/it-is-
possible-export-table-sqlite3-table-to-csv-or-similiar)

my table has 15 columns I just listed 5 here to make it easier to read

    
    
    writer = UnicodeWriter(open("Export8.csv", "wb"))
    
    writer.writerow(["RunID","JobNumber","StartTime","EndTime","Period"])
    
    writer.writerows(results)
    

# second attempt was:

    
    
    response = cursor.execute("SELECT RunID, JobNumber, StartTime, EndTime, strftime('%s',substr(endtime,1,19)) - strftime('%s',substr(starttime,1,19)) FROM tblTest WHERE RunID <>0")
    
    strfile = open('_output1.csv','wb')
    
    for row in response:
        print >> strfile,row
    

# third attempt was:

    
    
    strfile = open('_output3.csv','wb')
    
    while True:
    
        row = cursor.fetchone()
    
        if row == None:
    
            break
        print >> strfile,row
        enter code here
    

# 4th attempt/test:

    
    
    response = cursor.execute("SELECT RunID, JobNumber, StartTime, EndTime, Period FROM tblTest WHERE RunID <>0")
    
    print response
    

# Result

In attempt 1: I get an output of 183 full records and the very first column of
the 184 record

In attempt 2 and 3: I get an output of 181 full records and some columns of
the 182

In attempt 4: I get all my data on the screen

When i check the sqlite database I see 205 records. I am aware that I can just
output 100 lines at a time, but i am wondering why I am not getting all my
rows outputted


Answer: You can try using [pandas](http://pandas.pydata.org/) to load the sql data and
then to dump it into a csv. You'd have to install the dependencies (notably
NumPy) to use it though. It's really simple then:

    
    
    import sqlite3
    import pandas.io.sql as sql
    con = sqlite3.connect('database.db')
    table = sql.read_frame('select * from some_table', con)
    table.to_csv('output.csv')
    

