uploading records of list of files in parallel using python to DB
Question: I have a list of files each file have mass of records separting by \n , i need
to proccess those records in parallel and upload them to some sql server could
someone provide an idea what is the best way to do this with python


Answer: The best way might not be to upload in parallell but use SQL Servers bulk
importing mechanisims  
e.g.  
[BULK INSERT](http://msdn.microsoft.com/en-us/library/ms188365.aspx)  
[bcp](http://msdn.microsoft.com/en-us/library/ms162802.aspx)

EDIT:

If you need to process them then a way I have often used is  
1) bulk load the data into a staging table  
2) Process the data on the database  
3) Insert into main tables

Stages 2 and 3 can be combined if the processing is of a reasonable type.

This could be faster as there are less round trips to the server and
processing a set of data rather than row by row is usually quicker.

Also I thing that SQL server will make use of more than one CPU in doing this
processing so you get your processing parallel for free

