title: How can I combine multiple text files, remove duplicate lines and split the remaining lines into several files of certain length?
body: <p>I have a lot of relatively small files with about 350.000 lines of text.
  For example:</p>
  
  <p>File 1:</p>
  
  <pre><code> asdf
   wetwert
   ddghr
   vbnd
   ...
   sdfre
  </code></pre>
  
  <p>File 2:</p>
  
  <pre><code> erye
   yren
   asdf
   jkdt
   ...
   uory
  </code></pre>
  
  <p>As you can see line 3 of file 2 is a duplicate of line 1 in file 1.
  I want a program / Notepad++ Plugin that can check and remove these duplicates in multiple files.</p>
  
  <p>The next problem I have is that I want all lists to be combined into large 1.000.000 line files.
  So, for example, I have these files:</p>
  
  <ul>
  <li>648563 lines</li>
  <li>375924 lines</li>
  <li>487036 lines</li>
  </ul>
  
  <p>I want them to result in these files:</p>
  
  <ul>
  <li>1.000.000 lines</li>
  <li>511.523 lines</li>
  </ul>
  
  <p>And the last 2 files must consist of only unique lines.
  How can I possibly do this? Can I use some programs for this? Or a combination of multiple Notepad++ Plugins?
  I know GSplit can split files of 1.536.243 into files of 1.000.000 and 536.243 lines, but that is not enough, and it doesn't remove duplicates.</p>
  
  <p>I do want to create my own Notepad++ plugin or program if needed, but I have no idea how and where to start.</p>
  
  <p>Thanks in advance.</p>