title: How to EMR S3DistCp groupBy properly?
body: <p>I am using aws .net sdk to run a s3distcp job to EMR to concatenate all files in a folder with --groupBy arg. But whatever "groupBy" arg I have tried, it failed all the time or just copy the files without concatenating like if no --groupBy specified in the arg list.</p>
  
  <p>The files in the folder is spark saveAsTextFiles named like below:</p>
  
  <p>part-0000<br>
  part-0001<br>
  part-0002<br>
  ...<br>
  ...<br></p>
  
  <pre><code>step.HadoopJarStep = new HadoopJarStepConfig
              {
                  Jar = "/usr/share/aws/emr/s3-dist-cp/lib/s3-dist-cp.jar",
                  Args = new List&lt;string&gt;
                  {
                      "--s3Endpoint=s3-eu-west-1.amazonaws.com",
                      "--src=s3://foo/spark/result/bar" ,
                      "--dest=s3://foo/spark/result-merged/bar",
                      "--groupBy=(part.*)",
                      "--targetSize=256"
  
                  }
              };
  </code></pre>