title: Find duplicate entries in a text file using shell
body: <p>I am trying to find duplicate *.sh entry mention in a text file(test.log) and delete it, using shell program. Since the path is different so uniq -u always print duplicate entry even though there are two first_prog.sh entry in a text file</p>
  
  <pre><code>cat test.log       
  /mnt/abc/shellprog/test/first_prog.sh        
  /mnt/abc/shellprog/test/second_prog.sh    
  /mnt/abc/my_shellprog/test/first_prog.sh           
  /mnt/abc/my_shellprog/test/third_prog.sh    
  </code></pre>
  
  <p>output:    </p>
  
  <pre><code>/mnt/abc/shellprog/test/first_prog.sh    
  /mnt/abc/shellprog/test/second_prog.sh    
  /mnt/abc/my_shellprog/test/third_prog.sh    
  </code></pre>
  
  <p>I tried couple of way using few command but dont have idea on how to get above output.    </p>
  
  <pre><code>rev test.log | cut -f1 -d/ | rev | sort | uniq -d     
  </code></pre>
  
  <p>Any clue on this?    </p>