title: How to load objects from S3 bucket into Spark in RStudio?
body: <p>The Object in the S3 bucket is 5.3 GB size. In order to convert object into data, I used <code>get_object("link to bucket path")</code>. But this leads to memory issues.</p>
  
  <p>So, I installed Spark 2.3.0 in RStudio and trying to load this object directly into Spark but the command to load object directly into spark is not known.
  <code>library(sparklyr)
  library(dplyr)
  sc &lt;- spark_connect(master = "local")</code></p>
  
  <p>If I convert the object into a readable data type (such as data.frame/tbl in R) I would use <code>copy_to</code> to transfer the data into spark from R as below:</p>
  
  <h1>Copy data to Spark</h1>
  
  <p><code>spark_tbl &lt;- copy_to(spark_conn,data)</code></p>
  
  <p>I was wondering how can convert the object inside spark ?</p>
  
  <p>relevant links would be</p>
  
  <ol>
  <li><p><a href="https://github.com/cloudyr/aws.s3/issues/170" rel="nofollow noreferrer">https://github.com/cloudyr/aws.s3/issues/170</a></p></li>
  <li><p><a href="https://stackoverflow.com/questions/45190188/sparklyr-connection-to-s3-bucket-throwing-up-error">Sparklyr connection to S3 bucket throwing up error</a></p></li>
  </ol>
  
  <p>Any guidance would be sincerely appreciated.</p>