Python Pandas Large Row Processing
Row by row. Pandas is not the ideal tool for this.
I would suggest you look into Map/Reduce. It is designed for exactly this. Streaming is the key to row by row processing.
I have a lot of time series data. Almost 3 GB of csv files. The dimensions are 50k columns with 6000 rows. Now I need to process them row by row. They are time ordered and its important that for each row, I look at each column. 
Would importing this in to pandas as a pivot table and iterating them over row by row efficient? Any suggestions?