title: C++ - How to read Unicode characters( Hindi Script for e.g. ) using C++ or is there a better Way through some other programming language?
body: <p>I have a hindi script file like this:</p>
  <pre><code>3.  भारत का इतिहास काफी समृद्ध एवं विस्तृत है।
  </code></pre>
  <p>I have to write a program which adds a position to each and every word in each sentence.
  Thus the numbering for every line for a particular word position should start off with 1 in parentheses. The output should be something like this.</p>
  <pre><code>3.  भारत(1) का(2) इतिहास(3) काफी(4) समृद्ध(5) एवं(6) विस्तृत(7) है(8) ।(9)
  </code></pre>
  <p>The meaning of the above sentence is:</p>
  <pre><code>3.  India has a long and rich history.
  </code></pre>
  <p>If you observe the '।'( which is a full stop in hindi equivalent to a '.' in English ) also has a word position and similarly other special symbols would also have as I am trying to go about English-Hindi Word alignment( a part of Natural Language Processing ( NLP ) ) so the full stop in english '.' should map to '।'  in Hindi. Serial nos remain as it is untouched.
  I thought reading character by character could be a solution. How can I do this?</p>
  <p>The thing is I am able to get word positions for my English text using C++ as I was able to read character by character using ASCII values in C++ but I don't have a clue to how to go about the same for the hindi text.</p>
  <p>The final aim of all this is to see which word position of the English text maps to which postion in Hindi. This way I can achieve bidirectional alignment.</p>
  <p>Thank you for your time...:)</p>