Removing duplicate words from an string array and writing to an output file
<p>I have an assignment that requires me to take the words from an input.txt file and sort them by frequency and then write them to an output.txt file.</p>

<p>The problem I'm having is that once I find the frequency of each of the words from the input file, I have no idea how to remove the duplicates and how to keep them from being written to the output file. </p>

<p>Just to give a brief example, after sorting my input.txt, I get this written to my output.txt file:</p>

<p>a : 4</p>

<p>a : 3</p>

<p>a : 2</p>

<p>a : 1</p>

<p>and :   1</p>

<p>as :    4</p>

<p>as :    3</p>

<p>as :    2</p>

<p>as :    1</p>

<p>chuck : 5</p>

<p>chuck : 4</p>

<p>chuck : 3</p>

<p>chuck : 2</p>

<p>chuck : 1</p>

<p>The first line from each unique word is correct and I would like that to be the only thing printed to the output (ie a: 4 and: 1 as: 4 chuck: 5). My frequency is correct, so this is just a printing issue in removing the duplicates since they are obviously wrong. Thanks!</p>

<p>Here's the code:</p>

<pre><code>  import java.util.*;
  import java.io.*;


public class StringHistogram
{
  public static void main(String[] args) throws Exception
  {

     File fileOutput = new File("Output.txt");
     File file = new File("Input.txt");

     PrintWriter output = new PrintWriter(fileOutput);

     Scanner inputFile = new Scanner(file);
     Scanner outputFile = new Scanner(fileOutput);

     System.out.println("Program will now sort Input.txt to Output.txt...");


     StringBuilder sB = new StringBuilder();

     while(inputFile.hasNext())
     {
        String s = inputFile.nextLine();
        s = s + '\n';
        s = s.toLowerCase();
        s = s.replaceAll("[^a-z]", "\n");
        sB.append(s);

     }


     String[] tokens = sB.toString().split("[\n]+"); 
        Arrays.sort(tokens);

     for( int i=0; i&lt;tokens.length; i++)
     {
        int count = 1;
        for(int j = i+1; j &lt; tokens.length; j++)
        {
           if(tokens[i].compareTo(tokens[j]) ==0)
           {
              count++;
           }
           else
           {
              break;
           }

        }
        output.println(tokens[i] + " :\t" + count);
     }

     System.out.println("...Sorting is now complete! Check Output.txt");
     inputFile.close();
     output.close();


  }


}
</code></pre>
