Creating multiple files from a list This piece of code is to get random samples from two String lists, pos and neg. Based on a specified sampleSize, it writes an equal number of strings into 4 files. This codes seems slow, perhaps due to the file creating operation, since each file is created for a single string. Is there a better way to do this?

// pos and neg are ArrayList&lt;String&gt;
        Collections.shuffle(pos);
        Collections.shuffle(neg);

        int i = 0;
        int j =0;
        while(i &lt; sampleSize ){
            if(i&lt;sampleSize/2) {
                BufferedWriter bw1 = new BufferedWriter(new FileWriter(("test/neg/" + i + ".txt")));
                bw1.write(neg.get(i));
                bw1.close();

                BufferedWriter bw2 = new BufferedWriter(new FileWriter(("test/pos/" + i + ".txt")));
                bw2.write(pos.get(i));
                bw2.close();

            }else{
                BufferedWriter bw1 = new BufferedWriter(new FileWriter(("train/neg/" + j + ".txt")));
                bw1.write(neg.get(j));
                bw1.close();

                BufferedWriter bw2 = new BufferedWriter(new FileWriter(("train/pos/" + j + ".txt")));
                bw2.write(pos.get(j++));
                bw2.close();
            }
            i++;
            System.out.println(i);
        }
The execution time will be dominated by I/O, since you are writing so many files.  The only suggestion I have that might change the I/O performance is to try the NIO API, namely Files.write().

There is another possible inefficiency if sampleSize is much smaller than the size of the collections.  In that case, shuffling the entire list would be overkill, and you might be better off writing your own function to perform sampling without replacement.  (That would basically be a Fisher-Yates Shuffle that terminates early.)



There are four similar copies of the code, which you might want to consolidate, especially if it is not important to preserve the behaviour of your System.out.println(i) progress indicator.

My first observation is that the pos and neg lists can be treated independently.  My second observation is that you could simplify the loop if you interleave the output of the test and train cases.

I think that this code would make it more obvious that the number of files created is sampleSize, for each of the lists neg and pos.

private static void writeSampleFiles(List&lt;String&gt; list, String name, int sampleSize) throws IOException {
    Collections.shuffle(list);
    for (int i = 0; i &lt; sampleSize; i++) {
        Path path = Paths.get(i &amp; 1 == 0 ? "train" : "test",
                              name,
                              (i &gt;&gt; 1) + ".txt");
        Files.write(path, list.get(i).getBytes());
    }
}

// elsewhere...
writeSampleFiles(neg, "neg", sampleSize);
writeSampleFiles(pos, "pos", sampleSize);


Note that this isn't exactly identical to your original code.  The shuffled entries are extracted in a different order, which could be an issue if you are concerned about "dealing from the top of the deck".  If the execution aborts prematurely due to an IOException, then the difference in the processing order would be visible to the user.