Google App Engine Bulkloader "Unexpected Thread Death"
Question: I am trying to upload a moderetly sized csv file to google app engine using
the bulkloader functionality, and it appears to die some of the way through
with the following result:

    
    
    [INFO    ] Logging to bulkloader-log-20110328.181531
    [INFO    ] Throttling transfers:
    [INFO    ] Bandwidth: 250000 bytes/second
    [INFO    ] HTTP connections: 8/second
    [INFO    ] Entities inserted/fetched/modified: 20/second
    [INFO    ] Batch Size: 10
    [INFO    ] Opening database: bulkloader-progress-20110328.181531.sql3
    [INFO    ] Connecting to notmyrealappname.appspot.com/_ah/remote_api
    [INFO    ] Starting import; maximum 10 entities per post
    ...............................................................[INFO    ] Unexpected thread death: WorkerThread-7
    [INFO    ] An error occurred. Shutting down...
    .........[ERROR   ] Error in WorkerThread-7: <urlopen error [Errno -2] Name or service not known>
    
    [INFO    ] 1740 entites total, 0 previously transferred
    [INFO    ] 720 entities (472133 bytes) transferred in 32.3 seconds
    [INFO    ] Some entities not successfully transferred
    

It uploads about 700 of the 19k entries I am trying to upload, and I am
wondering why it fails. I checked the csv file for errors like additional
commas that could throw off the python csv reader and non ascii characters
have been stripped out.


Answer: Lifting the batch limit (batch_size) and rps limit (rps_limit) works, I use
1000 for the batch size and an rps limit of 500:

    
    
    appcfg.py upload_data --url= --application= --filename=  --email= --batch_size=1000 --rps_limit=500
    

