title: Spark RDD partitions while reading CSV from AWS S3
body: <p>I am unable to figure out how spark is deciding on number of partitions while reading from AWS S3</p>
  
  <p><strong>My Case:</strong></p>
  
  <p>I am using Spark 1.3 (sorry, but not in my hand)</p>
  
  <p>My <strong>S3 contains 100 csv files</strong> each of size ~60-75MB in batches i.e. folder1,folder2,folder3,etc contains 100 CSV files each</p>
  
  <p>I'm getting <strong>partitions 295-300</strong> while reading from this folders</p>
  
  <p>I'm  expecting that default partitions to be 200 always because if spark understands S3 data as a block-based system then it should read either 64MB or 128MB.</p>
  
  <p><em>Thanks in advance.</em></p>