Reading multiple csv files in a single dataframe
Ultimate optimization

avoid calling pd.DataFrame.append function within a loop as it'll create a copy of accumulated dataframe on each loop iteration. Apply pandas.concat to concatenate pandas objects at once.

no need to gzip.open as pandas.read_csv already allows on-the-fly decompression of on-disk data.

compression : {‘infer’, ‘gzip’, ‘bz2’, ‘zip’, ‘xz’, None}, default ‘infer’


avoid hardcoding filepathes with path+&quot;/&quot;+filename. Instead use suitable os.path.join feature: os.path.join(dirpath, fname)



The final optimized approach:
import os
import pandas as pd

dirpath = 'path_to_gz_files'   # your directory path
df = pd.concat([pd.read_csv(os.path.join(dirpath, fname))
                for fname in os.listdir(dirpath)], ignore_index=True)