title: R Parallelisation Error unserialize(socklisk[[n]])
body: <p>In a nutshell I am trying to parallelise my whole script over dates using Snow and adply but continually get the below error.</p>
  
  <pre><code>Error in unserialize(socklist[[n]]) : error reading from connection
  In addition: Warning messages:
  1: &lt;anonymous&gt;: ... may be used in an incorrect context: ‘.fun(piece, ...)’
  
  2: &lt;anonymous&gt;: ... may be used in an incorrect context: ‘.fun(piece, ...)’
  </code></pre>
  
  <p>I have set up the parallelisation process in the following way:</p>
  
  <pre><code>Cores = detectCores(all.tests = FALSE, logical = TRUE)
  cl = makeCluster(Cores, type="SOCK")
  registerDoSNOW(cl)
  clusterExport(cl, c("Var1","Var2","Var3","Var4"), envir = environment())
  
  
  exposureDaily &lt;- adply(.data = dateSeries,.margins = 1,.fun = MainCalcFunction,
                         .expand = TRUE, Var1, Var2, Var3, 
                         Var4,.parallel = TRUE)
  
  stopCluster(cl)
  </code></pre>
  
  <p>Where <code>dateSeries</code> might look something like</p>
  
  <pre><code>&gt; dateSeries
    marketDate
  1 2016-04-22
  2 2016-04-26
  </code></pre>
  
  <p><code>MainCalcFunction</code> is a very long script with multiple of my own functions contained within it. As the script is so long reproducing it wouldn't be practical, and a hypothetical small function would defeat the purpose as I have already got this methodology to work with other smaller functions. I can say that within <code>MainCalcFunction</code> I call all my libraries, necessary functions, and a file containing all other variables aside from those exported above so that I don't have to export a long list libraries and other objects.</p>
  
  <p><code>MainCalcFunction</code> can run successfully in its entirety over 2 dates using <code>adply</code> but not parallelisation, which tells me that it is not a bug in the code that is causing the parallelisation to fail.</p>
  
  <p>Initially I thought (from experience) that the parallelisation over dates was failing because there was another function within the code that utilised parallelisation, however I have subsequently rebuilt the whole code to make sure that there was no such function.</p>
  
  <p>I have poured over the script with a fine tooth comb to see if there was any place where I accidently didn't export something that I needed and I can't find anything.</p>
  
  <p>Some ideas as to what could be causing the code to fail are:</p>
  
  <ul>
  <li>The use of various option valuation functions in <code>fOptions</code> and <code>rquantlib</code></li>
  <li>The use of type <code>sock</code></li>
  </ul>
  
  <p>I am aware of <a href="https://stackoverflow.com/questions/7237024/error-in-unserialize-foreach-dosnow-snow-with-sock-windows">this</a> question already asked and also <a href="https://stackoverflow.com/questions/24327137/error-in-unserializesocklistn-error-reading-from-connection-on-unix">this</a> question, and while the first question has helped me, it hasn't yet help solve the problem. (Note: that may be because I haven't used it correctly, having mainly used <code>loginfo("text")</code> to track where the code is. Potentially, there is a way to change that such that I log warning and/or error messages instead?)</p>
  
  <p>Please let me know if there is any other information I can provide to help in solving this. I would be so appreciative if someone could provide some guidance, as the code takes close to 40 minutes to run for a day and I need to run it for close to a year, therefore parallelisation is essential!</p>
  
  <p><strong>EDIT</strong></p>
  
  <p>I have tried to implement the suggestion in the first question included above by utilising the outfile option. Given I am using Windows, I have done this by including the following lines before the exporting of the key objects and running <code>MainCalcFunction</code> :</p>
  
  <pre><code>reportLogName &lt;- paste("logout_parallel.txt", sep="")
  addHandler(writeToFile,  
             file = paste(Save_directory,reportLogName, sep="" ),
             level='DEBUG')
  with(getLogger(), names(handlers))
  
  loginfo(paste("Starting log file", getwd()))
  
  mc&lt;-detectCores()
  cl&lt;-makeCluster(mc, outfile="")  
  registerDoParallel(cl)
  </code></pre>
  
  <p>Similarly, at the beginning of <code>MainCalcFunction</code>, after having sourced my libraries and functions I have included the following to print to file:</p>
  
  <pre><code>reportLogName &lt;- paste(testDate,"_logout.txt", sep="")
      addHandler(writeToFile,  
                 file = paste(Save_directory,reportLogName, sep="" ),
                 level='DEBUG')
      with(getLogger(), names(handlers))
  
      loginfo(paste("Starting test function ",getwd(), sep = ""))
  </code></pre>
  
  <p>In the <code>MainCalcFunction</code> function I have then put <code>loginfo("text")</code> statements at key junctures to inform me of where the code is at. </p>
  
  <p>This has resulted in some text files being available after the code fails due to the aforementioned error. However, these text files provide no more information on the cause of the error aside from at what point. This is despite having a <code>tryCatch</code> statement embedded in <code>MainCalcFunction</code> where at the end, on any instance of error I have added the line <code>logerror(e)</code></p>