title: Why is Polybase slow for large compressed files that span 1 billion records?
body: <p>What would cause Polybase performance to degrade when querying larger datasets in order to insert records into Azure Data Warehouse from Blob storage?</p>
  
  <p>For example, a few thousand compressed (.gz) CSV files with headers partitioned by a few hours per day across 6 months worth of data. Querying these files from an external table in SSMS is not exactly optimial and it's extremely slow.</p>
  
  <p>Objectively, I'm loading data into Polybase in order to transfer data into Azure Data Warehouse. Except, it seems with large datasets, Polybase is pretty slow.</p>
  
  <p>What options are available to optimize Polybase here? Wait out the query or load the data after each upload to blob storage incrementally?</p>