CDF with a massive amount of data I am trying to plot a CDF of 7 files I have. Each file looks like this:

1
2
1.5
2.3


and so on. The problem is about the sizes of the files, which are:

862M
1,8G
2,4G
18G
2,0G
1,8G


I have put up a simple R script which simply loads the files and plot them. When I generate fake files of smaller size, the script works fine. However, it is running since three days and it hasn't produced any output using the full files.

The script is this:

library(ggplot2)    
data &lt;- read.table('file1.csv')
data$g = "G1"
data2 &lt;- read.table('file2.csv')
data2$g = "G2"
data3 &lt;- read.table('file3.csv')
data4$g = "G3"
data4 &lt;- read.table('file4.csv')
data4$g = "G4"
data5 &lt;- read.table('file5.csv')
data5$g = "G5"
data6 &lt;- read.table('file56.csv')
data6$g = "G6"
data7 &lt;- read.table('file7.csv')
data7$g = "G7"

dftotal = rbind(data,data2)
dftotal = rbind(dftotal,data3)
dftotal = rbind(dftotal,data4)
dftotal = rbind(dftotal,data5)
dftotal = rbind(dftotal,data6)
dftotal = rbind(dftotal,data7)

gp &lt;- ggplot(data = dftotal, aes(x = V1), group = factor(g)) + stat_ecdf()
ggsave('cdf.eps',gp)


Does anyone knows a more efficient way to do this?
My estimate is that there are around 5.4b numbers in your dataset, all in the range of 0-50 according to your comment. I doubt there are much faster ways to do this on all the data than you are currently doing. However if you would just take every file and get random samples of 0.1-10% of each file and then combine these, you will still get an ecdf that is visually almost exactly the same while reducing memory issues and computing power necessary.