be2f37df_er_Checklist___SearchTools_com__Action
[Content] Handling Multiple Languages [Action] 
Indexing text works fairly well even if you don't know much about the language. The more information you can add regarding word breaks and punctuation, the better it works, so consider designing your system in ways that allow future extensions in this area. HTML has a way for page creators to specify the character encoding in the Content-Type header field.
Indexing Robot Crawler Checklist - SearchTools.com
Action