title: Non-English Language Synonym Filter Factory for SOLR
body: <p>I have been trying to use the synonym.txt file and the SynonymFilterFactory that ships out of the box with SOLR Lucene, with Indian Languages (Hindi for POC) but it doesn't seem to work as it works for English.<br/></p>
  
  <p>Found <a href="https://stackoverflow.com/questions/27128070/solr-russian-synonyms-are-not-working?rq=1">this</a> here on stack overflow which raises a similar question but has no resolution, yet.</p>
  
  <p>I have already taken care of the following to support Indian Language Search with SOLR,<br/>
  1. Changed Browser Encoding to UTF-8<br/>
  2. Added URIEncodings=UTF-8 in server.xml of Acapche Tomcat Server.<br/></p>
  
  <p>For the POC I have tried out the following things,<br/>
  1. Created a new Field Type to support Hindi indexing,</p>
  
  <pre><code>&lt;fieldType name="text_hi" class="solr.TextField" positionIncrementGap="100"&gt;
    &lt;analyzer&gt; 
      &lt;tokenizer class="solr.StandardTokenizerFactory"/&gt;
      &lt;filter class="solr.LowerCaseFilterFactory"/&gt;
      &lt;!-- normalizes unicode representation --&gt;
      &lt;filter class="solr.IndicNormalizationFilterFactory"/&gt;
      &lt;!-- normalizes variation in spelling --&gt;
      &lt;filter class="solr.HindiNormalizationFilterFactory"/&gt;
      &lt;filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_hi.txt" /&gt;
      &lt;filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/&gt;
      &lt;filter class="solr.HindiStemFilterFactory"/&gt;
    &lt;/analyzer&gt;
  &lt;/fieldType&gt;
  </code></pre>
  
  <p><strong>UPDATE</strong>
  I also tried removing the stemming after going through the responses by @Mysterion and @Alexandre Rafalovitch on this post,</p>
  
  <pre><code>&lt;fieldtype name="text_hi_rev" class="solr.TextField" positionIncrementGap="100"&gt;
  &lt;analyzer&gt;
    &lt;tokenizer class="solr.WhitespaceTokenizerFactory"/&gt;
    &lt;filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true" tokenizerFactory="solr.WhitespaceTokenizerFactory"/&gt;
  &lt;/analyzer&gt;
  </code></pre>
  
  <p></p>
  
  <ol start="2">
  <li><p>Defined a new Field based on the created field type,<br/></p>
  
  <p></p></li>
  <li><p>Added the following line to the synonyms.txt file,<br/></p>
  
  <p>india,bharat,भारत , हिन्दुस्तान ,hindustan</p></li>
  <li><p>Indexed the following strings as part different documents,<br/></p>
  
  <p>मैं <strong>भारत</strong> का रहने वाला हूँ
  मैं <strong>हिसंदुस्तान</strong> का रहने वाला  हूँ 
  मैं <strong>india</strong> का रहने वाला हूँ
  मैं <strong>hindustan</strong> का रहने हूँ
  मैं <strong>bharat</strong> का रहने हूँ</p></li>
  </ol>
  
  <p><strong>Expected Behaviour</strong> : <br/>
  When I search for any of the keywords <strong>india,bharat,भारत , हिन्दुस्तान ,hindustan</strong> I should get all the documents indexed in Step-4<br/></p>
  
  <p><strong>Actual Behaviour</strong>:<br/>
  1. When searching with keywords <strong>india, hindustan,or bharat</strong> I get the following results,<br/>
  मैं <strong>india</strong> का रहने वाला हूँ
  मैं <strong>hindustan</strong> का रहने हूँ
  मैं <strong>bharat</strong> का रहने हूँ</p>
  
  <ol start="2">
  <li>Searching with <strong>भारत</strong> returns the document with <strong>भारत</strong> ONLY and same with 
  <strong>हिसंदुस्तान</strong><br/></li>
  </ol>
  
  <p>Any pointers if what I am trying is even possible? If possible, what could I be doing wrong here?<br/></p>
  
  <p>Thanks.</p>