What is the fastest way to stream a large csv file?
In pandas.read_csv you can use the "chunksize" option, if you do, the object returned by pandas will be an iterator (of type TextFileReader) which when iterated over will return a DataFrame reading over number_of_rows<=chunksize (I hadn't realized the option existed until I read the source code...).
I've compared the built-in csv reader with Pandas's read_csv. The former is significantly slower. However, I have a need to stream csv files due to memory limitation. What streaming csv reader that is as fast or almost as fast as Pandas?