title: How to load and filter data efficiently in SSIS
body: <p>Need to loading a flat file with an SSIS Package executed in a scheduled job in SQL Server 2016 but it's taking TOO MUCH TIME (like 2/3 hours) just to load data in source then it’s need extra (2/3 hours) time for sort and filter then need similar time to load data in target, the file just has like million rows and it’s not less than 3 GB file approximately. This is driving me crazy, because is affecting the performance of my server.</p>
  
  <p><strong>SSIS package:</strong> -My package is just a Data Flow Task that has a Flat File Source and an OLE DB Destination, that’s all -The Data Access Mode is set to FAST LOAD. -Just have 1 indexes in the table. My destination table has 32 columns </p>
  
  <p><strong>Input file:</strong>
  Input text file has more than 32 columns, surrogate key data may not unique , referenced columns date may not unique , Need to filter them.</p>
  
  <p>Face <strong>two</strong> problems one is SSIS FlatFile-Source take huge time to load date another one is sort and filter. What to do?</p>
  
  <p><a href="https://i.stack.imgur.com/9B4vB.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/9B4vB.png" alt="enter image description here"></a></p>