title: should I pre-install cran r packages on worker nodes when using sparkr
body: <p>I want to use r packages on cran such as <code>forecast</code> etc with sparkr and meet following two problems.</p>
  
  <ol>
  <li><p>Should I pre-install all those packages on worker nodes? But when I read the source code of spark <a href="https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/RPackageUtils.scala" rel="nofollow">this file</a>, it seems that spark will automatically zip packages and distribute them to the workers via --jars or --packages. What should I do to make the dependencies available on workers?</p></li>
  <li><p>Suppose I need to use functions provided by <code>forecast</code> in a <code>map</code> transformation, how should I import the package. Do I need to do something like following, import the package in the map function, will it make multiple import:
  <code>
  SparkR:::map(rdd, function(x){
    library(forecast)
    then do other staffs
  })
  </code></p></li>
  </ol>
  
  <p>Update:</p>
  
  <p>After reading more source code, it seems that, I can use <code>includePackage</code> to include packages on worker nodes according to <a href="https://github.com/apache/spark/blob/master/R/pkg/R/context.R" rel="nofollow">this file</a>. So now the problem becomes is it right that I have to pre-install the packages on nodes manually? And if that's true, what's the use case for --jars and --packages described in question 1? If that's wrong, how to use --jars and --packages to install the packages?</p>