title: How to read millions of rows(575.7 MB) from AWS S3 gzipped file to Pandas DataFrame?
body: <p>The file is copied into S3 from <strong>Snowflake</strong> using COPY INTO in airflow. It is creating a <strong>gzip</strong> file to S3(but with .csv extension) and I would like to read it into DataFrame.
  I've tried numerous ways(With gzip.GzipFile(), Dask, pd.read_csv(compression='gzip', loading in chunks) but it takes too long and Airflow will exit with an error: <code>Task exited with return code Negsignal.SIGKILL</code> (Most of those solutions won't even end loading in Jupyter).</p>
  <p>What would be the most effective way to do this?</p>
  <p>P.S I don't really need the file from Snowflake to S3 but just to a Dataframe(from table) and I've also tried getting chunks of data from Snowflake using <code>Snowflakehook.get_pandas_df(query='SELECT * FROM table limit {} offset {})</code> append those sub_df's to a list and then <strong>pd.concat</strong> them together but this will also exit with SIGKILL after getting only 1 million rows.</p>
  <p>Thanks in advance.</p>