I have two files each with the same ([Hindi](https://en.wikipedia.org/wiki/Hindi)) word but I copied the word for each file from different sources. While the words from both the sources are alike visually, their bytes are different. The files are [here](http://www.wikiupload.com/MAGPIM65C3DWHKW) and [here](http://www.wikiupload.com/ELHKFUFACH6AP5T). I am not sure about the original encoding in both cases but opening the file as UTF-8 displays the characters correctly. 

It is also interesting that when I do unique using the uniq utility only one entry is returned however when I place them in a file and did sort u in vim, I get both the entries.

Please explain what's going on.

Update: 

If you do not want to open the links, the Python literals: `'\u091c\u0941\u095c\n'` and `'\u091c\u0941\u0921\u093c\n'` and the word looks like

[![hindi](https://i.stack.imgur.com/plC61.png)](https://i.stack.imgur.com/plC61.png)