R random forest on Amazon ec2 Error: cannot allocate vector of size 5.4 Gb I am training random forest models in R using randomForest() with 1000 trees and data frames with about 20 predictors and 600K rows. On my laptop everything works fine, but when I move to amazon ec2, to run the same thing, I get the error:

Error: cannot allocate vector of size 5.4 Gb
Execution halted


I'm using the c3.4xlarge instance type so it's pretty beefy. Does anyone know a workaround for this to get it to run on this instance? I would love to know the memory nuances that causes this problem only on the ec2 instance and not on my laptop (OS X 10.9.5 Processor  2.7 GHz Intel Core i7; Memory  16 GB 1600 MHz DDR3)

Thanks.
Here's some advice (use at your own risk!):


make sure that your R environment on EC2 is identical to the one on your laptop
make sure you're using 64-bit images for your virtual machine instance
try to create/enable swap space: https://stackoverflow.com/a/22247782/2872891
see this discussion: https://stackoverflow.com/q/5171593/2872891
see this discussion: https://stackoverflow.com/q/12322959/2872891
see this discussion: https://stackoverflow.com/q/1358003/2872891


If the above-mentioned simpler measures don't help OR you want to achieve more scalability and/or performance, including an ability to parallelize the process on a single machine or across multiple machines, consider using bigrf R package: http://cran.r-project.org/web/packages/bigrf. Also see this discussion: https://stackoverflow.com/q/1358003/2872891.