title: unserialize error in sparkR
body: <p>I am very new to SparkR (and parallelization in general). I am running SparkR locally (I know that is not the right usage of spark but I am just getting started) and I have tried to re-write some part of my code with sparkR though 
  collect gives me the following errors by increasing the number of samples as (no error for small number of samples):</p>
  
  <pre><code>Error in unserialize(obj) : 
  ReadItem: unknown type 0, perhaps written by later version of R
  Calls: assetForecast ... convertJListToRList -&gt; lapply -&gt; lapply -&gt; FUN   -&gt; unserialize
  Execution halted
  </code></pre>
  
  <p>and the other error which is is probably because of my low memory is:</p>
  
  <pre><code>heap memory error (trying increasing JVM memory &amp; driver memory did not help) 
  </code></pre>
  
  <p>I would appreciate any help regarding FIRST error (I posted the second error since I thought they may be somehow related even though I get them by setting different values for numSlices in parallelize). I think the first one may be a version incompatibility between spark, sparkR and R that causes this serialization issue. I tried installing different version though pretty soon stuck with resolving dependency.</p>
  
  <p>Here is a sample script which simulates what I am doing in SparkR (the error are generated for input.len > 950):</p>
  
  <pre><code>library(SparkR) # load sparkR library
  sc &lt;- sparkR.init() ## initialize the sparkR
  input.len &lt;- 8000 # size of the input
  num.slice &lt;- 2 # number of slices for parallelize function
  
  ## Define a few functions to simulate actual calculations
  latemail &lt;- function(N, st="2012/01/01", et="2015/12/31") {
     ## create random date of length N
     st &lt;- as.POSIXct(as.Date(st))
     et &lt;- as.POSIXct(as.Date(et))
     dt &lt;- as.numeric(difftime(et,st,unit="sec"))
     ev &lt;- sort(runif(N, 0, dt))
     rt &lt;- st + ev
  }
  
  encode &lt;- function(ele1, ele2) {
     ## concatenate ele1 and ele2, seperated by %
     return (paste(toString(ele1), toString(ele2), sep = "%"))
  }
  
  decode &lt;- function(coded) {
     ## separate input string by %
     idx &lt;- regexpr("%", coded)[1]
     ele1 &lt;- as.numeric(substr(coded, 1, idx-1))
     ele2 &lt;- substr(coded, idx + 1, nchar(coded))
     return (list(ele1, ele2))
   }
  
   fakeFun &lt;- function(asset.age, asset.year) {
      ## fake function to simulate my actual function
      return (as.list(rep(asset.age, 10)))
   }
  
    wrapperFun &lt;- function(x) {
       asset.age &lt;- decode(x)[[1]]
       asset.y &lt;- decode(x)[[1]]
       df &lt;- fakeFun(asset.age, asset.y)
       return (df)
    }
  
    ## Start of calculations with SparkR
    calc.ts &lt;- latemail(input.len) ## create fake years
    asset.ages &lt;- runif(input.len) * 10 ## create fake ages
  
    paired &lt;- list()
    for (i in 1:length(asset.ages)) {
       ## keep information of both years and ages in one vector
       ## using encode function
       paired[[length(paired) + 1]] &lt;- encode(asset.ages[[i]], calc.ts[[i]])
    }
  
    rdd.paired &lt;- parallelize(sc, paired, numSlices = num.slice)
    rdd.df &lt;- lapply(rdd.paired, wrapperFun)
    rdd.list &lt;- collect(rdd.df)
    print(rdd.list)
    sparkR.stop()
  </code></pre>
  
  <p>Here is the full report of error: </p>
  
  <pre><code>for numSlice = 5 in parallelize function:
      &gt; rdd.list &lt;- collect(rdd.df)
      15/07/22 17:20:40 INFO RRDD: Times: boot = 0.434 s, init = 0.015 s, broadcast = 0.000 s, read-input = 0.003 s, compute = 0.200 s, write-output = 0.004 s, total = 0.656 s
      15/07/22 17:20:41 INFO RRDD: Times: boot = 0.010 s, init = 0.017 s, broadcast = 0.000 s, read-input = 0.003 s, compute = 0.193 s, write-output = 0.004 s, total = 0.227 s
      15/07/22 17:20:41 INFO RRDD: Times: boot = 0.010 s, init = 0.013 s, broadcast = 0.001 s, read-input = 0.002 s, compute = 0.191 s, write-output = 0.003 s, total = 0.220 s
      15/07/22 17:20:41 INFO RRDD: Times: boot = 0.010 s, init = 0.011 s, broadcast = 0.000 s, read-input = 0.002 s, compute = 0.191 s, write-output = 0.004 s, total = 0.218 s
      15/07/22 17:20:41 INFO RRDD: Times: boot = 0.014 s, init = 0.015 s, broadcast = 0.000 s, read-input = 0.003 s, compute = 0.213 s, write-output = 0.004 s, total = 0.249 s
      Error in unserialize(obj) : 
        ReadItem: unknown type 0, perhaps written by later version of R
      Calls: collect ... convertJListToRList -&gt; lapply -&gt; lapply -&gt; FUN -&gt; unserialize
      Execution halted
  
  for numSlice = 6 in parallelize function
  15/07/22 17:18:52 WARN TaskSetManager: Lost task 2.0 in stage 0.0 (TID 2, localhost): java.lang.OutOfMemoryError: Java heap space
          edu.berkeley.cs.amplab.sparkr.RRDD.readData(RRDD.scala:258)
          edu.berkeley.cs.amplab.sparkr.RRDD.readData(RRDD.scala:243)
          edu.berkeley.cs.amplab.sparkr.BaseRRDD.read(RRDD.scala:200)
          edu.berkeley.cs.amplab.sparkr.BaseRRDD$$anon$1.next(RRDD.scala:70)
          scala.collection.Iterator$class.foreach(Iterator.scala:727)
          edu.berkeley.cs.amplab.sparkr.BaseRRDD$$anon$1.foreach(RRDD.scala:66)
          scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
          scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
          scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
          scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
          edu.berkeley.cs.amplab.sparkr.BaseRRDD$$anon$1.to(RRDD.scala:66)
          scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
          edu.berkeley.cs.amplab.sparkr.BaseRRDD$$anon$1.toBuffer(RRDD.scala:66)
          scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
          edu.berkeley.cs.amplab.sparkr.BaseRRDD$$anon$1.toArray(RRDD.scala:66)
          org.apache.spark.rdd.RDD$$anonfun$16.apply(RDD.scala:774)
          org.apache.spark.rdd.RDD$$anonfun$16.apply(RDD.scala:774)
          org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121)
          org.apache.spark.SparkContext$$anonfun$runJob$4.apply(SparkContext.scala:1121)
          org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:62)
          org.apache.spark.scheduler.Task.run(Task.scala:54)
          org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:177)
          java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          java.lang.Thread.run(Thread.java:745)
  15/07/22 17:18:52 ERROR TaskSetManager: Task 2 in stage 0.0 failed 1 times; aborting job
  Error in readTypedObject(con, type) : 
    Unsupported type for deserialization 
  Calls: collect ... callJMethod -&gt; invokeJava -&gt; readObject -&gt; readTypedObject
  Execution halted
  </code></pre>
  
  <p>Is there really a problem in my SparkR installation? If yes, how it runs for small number of samples?</p>
  
  <p>Thanks a lot</p>