title: Best way to transfert data from RDBMs to S3 by batch
body: <p>I have a requirement to transfer data (frequency = one transfer every evening) from on-prem relational database (SQL server) to AWS S3 (As csv files and csv format is a requirement). The data size is around 500 GB. Given the absence of CDC, the entire database will be copied every evening. </p>
  
  <p>Of course, we can think of formats like parquet or orc, but csv is a requirement in our use-case. </p>
  
  <p>I thought to a JDBC link with an EMR-spark cluster to import data from the on-prem RDBMs database before transforming it into a csv and sending it to S3, but I am not sure that it is the most optimal architecture.</p>
  
  <p>For JDBC, i do not know if there is a way to have several connections (btw emr-spark and sqlserver) in parallel to speed up the transfer.</p>
  
  <p>It is a VPN connection which connects the on-prem site and the aws part. </p>
  
  <p>Can someone suggest me the best way to have the fastest transfer respecting the principles of aws well-architected ?  </p>