title: Fix Japanese character issue in unix box
body: <p>Our company build process uses gradle and copies over the deliverables including sql file to a shared drive. Unix boxes can refer to these paths and then uses a shell script. The shell script invokes the db2 command line utility to run the sql files in the corresponding environment's database.</p>
  
  <p>This works pretty well. However I have hit a wall where we are updating japanese characters in the sql file. When I download the file from the browser and inspect the file, I can see the characters properly. However, in the unix box, I can see that the content in the sql file looks pretty different. I used cat, more and vi to view the contents of the file. Value of the LANG environment variable is C. I see the environment variable set is TERM=XTERM. So I think xterm is the terminal evaluator.</p>
  
  <pre><code>LANG=C
  </code></pre>
  
  <p>When I run the command in the linux terminal window-</p>
  
  <pre><code>locale charmap
  </code></pre>
  
  <p>I see the encoding as ANSI_X3.4-1968. UTF-8 is one of the possiblities but that is not set in the box I think.</p>
  
  <p>When I run the command in the linux terminal -</p>
  
  <pre><code>file -bi 1.sql
  </code></pre>
  
  <p>I see the charset encoding of the file as UTF-8. </p>
  
  <p>When we run the shellscript which invokes the db2 command line utility, it inserts whatever I can see in the unix file and not the actual japanese characters.</p>
  
  <p>I tries searching for options as to what could go wrong. Please find below my analysis - </p>
  
  <ol>
  <li><p>When I run the sql from the Aqua Data Studio, data is inserted properly. So there is no issue with the actual database setup or the table configuration.</p></li>
  <li><p>There are some other xml files in the shared path which have japanese characters. In unix box, I again see the distorted characters as I see in the sql file. When these xml files get copied to the reporting server, they work fine and the data is shown properly in the pdf.</p></li>
  </ol>
  
  <p>So my line of thinking is that there can be some way I can specify in the db2 command line utility to use the correct character set and that should work fine. But I am unable to find anything useful on the internet.</p>
  
  <p>Now there are chances that there is something small which I am not able to figure out. Hence I will request you to help me in this regard. This is not very critical as we can always leave the automated process of deployment and sql insertion , and instead go with the manual insertion by giving the scripts to the DBA. However this is highly discouraged.</p>
  
  <p>Do let me know if you could help me with this regard. Just FYI, I unzipped the war produced after the gradle build (from the jenkins server). I can see the file properly in the notepad++. Do let me know if you need any other info.</p>