title: Creating an ETL system (Data import and transformation)
body: <p>I have been tasked to write a module for importing data into a client's system. 
  I thought to break the process into 4 parts:  </p>
  
  <p><strong>1.</strong> Connect to the data source (SQL, Excel, Access, CSV, ActiveDirectory, Sharepoint and Oracle) - DONE<br>
  <strong>2.</strong> Get the available tables/data groups from the source - DONE<br>
  &nbsp;&nbsp;&nbsp;&nbsp;i. Get the available fields form the selected table/data group - DONE<br>
  &nbsp;&nbsp;&nbsp;&nbsp;ii. Get all data from the selected fields - DONE<br>
  <strong>3.</strong> Transform data to the user's requirements<br>
  <strong>4.</strong> Write the transformed data the the MSSQL target</p>
  
  <p>I am trying to plan how to handle complex data transformations like:<br>
  Get column A from Table tblA, inner joined to column FA from table tblB, and concatenate these two with a semicolon in between.</p>
  
  <p>OR  </p>
  
  <p>Get column C from table tblC on source where column tblC.D is not in table tblG column G on target database.</p>
  
  <p>My worry is not the visual, but the representation in code of this operation.</p>
  
  <p>I am NOT asking for sample code, but rather for some creative ideas.</p>
  
  <p>The data transformation will not be with free text, but drag and drop objects that represent actions.</p>
  
  <p>I am a bit lost, and need some fresh input.</p>