title: Removing duplicate lines from a text file
body: <p>I have a text file that is sorted alphabetically, with around 94,000 lines of names (one name per line, text only, no punctuation.</p>
  
  <p>Example:</p>
  
  <p>Alice</p>
  
  <p>Bob</p>
  
  <p>Simon</p>
  
  <p>Simon</p>
  
  <p>Tom</p>
  
  <p>Each line takes the same form, first letter is capitalized, no accented letters.</p>
  
  <p>My code:</p>
  
  <pre><code>try{
          BufferedReader br = new BufferedReader(new FileReader("orderedNames.txt"));
          PrintWriter out = new PrintWriter(new BufferedWriter(new FileWriter("sortedNoDuplicateNames.txt", true)));
  
          ArrayList&lt;String&gt; textToTransfer = new ArrayList();
  
  
          String previousLine = "";
          String current = "";
  
          //Load first line into previous line
          previousLine = br.readLine();
  
          //Add first line to the transfer list
          textToTransfer.add(previousLine);
  
  
          while((current = br.readLine()) != previousLine &amp;&amp; current != null){
  
              textToTransfer.add(current);
              previousLine = current;
          }
          int index = 0;
          for(int i=0; i&lt;textToTransfer.size(); i++){
              out.println(textToTransfer.get(i));
              System.out.println(textToTransfer.get(i));
              index ++;
  
          }
          System.out.println(index);
  
  }catch(Exception e){
      e.printStackTrace();
  }
  </code></pre>
  
  <p>From what I understand is that, the first line of the file is being read and loaded into the previousLine variable like I intended, current is being set to the second line of the file we're reading from, current is then compared against the previous line and null, if it's not the same as the last line and it's not null, we add it to the array-list.</p>
  
  <p>previousLine is then set to currents value so the next readLine for current can replace the current 'current' value to continue comparing in the while loop.</p>
  
  <p>I cannot see what is wrong with this.
  If a duplicate is found, surely the loop should break?</p>
  
  <p>Sorry in advance when it turns out to be something stupid.</p>