What do you need kanatype sensitivity for?


Question to any Japanese reader:

When you look in SQL-Server and do

[code]

    SELECT * FROM sys.fn_helpcollations() 
    WHERE (1=1) 
    --AND description LIKE '%insensitive%'
    AND name LIKE '%japanese%' 
    
[/code]

you get Kanatype-sensitive and Kanatype-insensitive collations.

My understanding is, that this is two different kind of alphabets, Hiragana
and Katakana.

SQL-Server documentation states:

> If this option is not selected, SQL Server considers Hiragana and Katakana
> characters to be equal for sorting purposes

So my question:

When/Why would you want to use kanatype-insensitive collation?  
What's the deeper purpose?  
Is a phonebook-lookup kanatype-sensitive or kanatype-insensitive?

In which cases would you use a sensitive collation, and when an insensitive
collation?




In general, if you're storing any Japanese text that needs to be sorted, you
probably want to go with Kanatype insensitive. Why would you want it like
this? Because it makes sorting more intuitive in terms of Japanese language.

In english, since we have only one writing system, it's easy to sort things
algorithmically. We simply order the characters by their character codes
(already in alphabetical order) and we're done. In Japanese, though, because
there are multiple ways to write out equivalent sounds, sorting can get a bit
tricky. Hiragana and Katakana alphabets are separated into separate Unicode
blocks, so when we try sorting things with "Kanatype sensitivity", we end up
with results that aren't completely intuitive.

Imagine you had a list of names that you wanted to sort:

[code]

    { "ピカチュウ","さとし","マリオ","まちだ","はるか" }
    
[/code]

The romanized equivalent to the list is:

[code]

    { "Pikachu","Satoshi","Mario","Machida","Haruka" }
    
[/code]

When sorted kanatype sensitive, you would get the following result:

[code]

    { "さとし","はるか","まちだ","ピカチュウ","マリオ" }
    { "Satoshi","Haruka","Machida","Pikachu","Mario" }
    
[/code]

When sorted kanatype insensitive, you would get this result instead:

[code]

    { "さとし","はるか","ピカチュウ","まちだ","マリオ" }
    { "Satoshi","Haruka","Pikachu","Machida","Mario" }
    
[/code]

To Japanese speakers, the second sort is a lot more intuitive, as the results
are actually sorted phonetically instead of based on character sets. "まちだ" and
"マリオ" both start with the same phonetic sound, but because one uses hiragana
"ma" and the other uses katakana "ma", they are separated when kanatype
sensitivity is enabled. With kanatype insensitivity, the list can be properly
sorted so that the two words appear next to each other on the list despite
their writing system differences.

A good analogy for English language would be case-sensitivity. Imagine if you
wanted to sort a list of words for a dictionary, some of them proper nouns
while others are not:

[code]

    {"New York","new","jet","Japan","squirm","SQL"}
    
[/code]

If we ignored the fact that uppercase and lowercase letters represent the same
letter and just sort based on character code, we would get something like
this:

[code]

    {"Japan", "New York", "SQL", "jet", "new", "squirm"}
    
[/code]

A dictionary sorted like this would hardly be useful, especially if we wanted
to look up a word without knowing whether it started with an uppercase or
lowercase letter. We'd have to check the first part of the dictionary with all
the proper nouns before checking the last part with all other words.

If we ran a case insensitive sort that treat "A" and "a" as the same letter
despite having separate character codes. We would get a result that is much
more intuitive:

[code]

    {"Japan","jet","new","New York","squirm","SQL"}
    
[/code]

So in general, unless you have a specific reason not to, you should always
disable kanatype sensitivity. A phonebook-lookup would be kanatype sensitive.
Note that in Japanese there is also an additional character type, Kanji, that
you would also need to work with. Kanji is much harder to sort, as there are
almost always multiple ways to read each Kanji and no real "alphabetical"
order to the Kanji. Most forms intended for Japanese people usually have two
fields for names: the user's name as it is normally written out, and the
user's name completely written out in katakana. Not only does this let people
know how to correctly pronounce a name which might be ambiguous written solely
in Kanji, but it allows software to sort by the unambiguous katakana-only
field, making the sort kanatype insensitive.

For more information, I definitely recommend checking out [this excellent
article](http://www.localizingjapan.com/blog/2011/02/13/sorting-in-
japanese-%E2%80%94-an-unsolved-problem/), which explains the issues with
sorting in Japanese much better than I can.

**TL;DR:** Kanatype insensitivity makes sorting Japanese text more intuitive
and should generally always be enabled unless you have a reason not to.

