gram. In the given example, “x y” is the context. The left most token in the context is referred to as the left token. The future token is the last token of the n-gram, which in the example is “z”. The n-gram can also be described with respect to a right context and a backed off context. The right context includes all tokens of the n-gram following the first token of the n-gram, represented as a (n−1)-gram. In the example above, “y z” is the right context. Additionally, the backed off context is the context of the n-gram less the left most token in the context. In the example above, “y” is the backed off context.
Each n-gram has an associated probability estimate that is calculated as a function of n-gram relative frequency in training data. For example, a string of L tokens is represented as C1L=(c1, c2,..., cL). A probability can be assigned to the string C1L as:
            P      ⁡              (                  c          1          L                )              =                            ∏                      i            =            1                    L                ⁢                  P          ⁡                      (                                          c                i                            ❘                              c                1                                  i                  -                  1                                                      )                              ≈                        ∏                      i            =            1                    L                ⁢                              P            ^                    ⁡                      (                                          c                i                            ❘                              c                                  i                  -                  n                  +                  1                                                  i                  -                  1                                                      )                               ,where the approximation is based on a Markov assumption that only the most recent (n−1) tokens are relevant when predicting a next token in the string, and the “^” notation for P indicates that it is an approximation of the probability function.
In CJK languages, sentences do not have word boundaries. As a result, sentences need to be segmented automatically before the detection of people's names. Therefore, segmentation errors will be propagated to name detection.
CJK names have morphologic laws that can be obtained from large statistics. For example, 300 common Chinese family names cover 99% or more of the population. Female names often contain characters such as  (na, hong, bing, li). Usually, common given names are independent of family names. For example, if
gram. In the given example, “ x y ” is the context. The left most nominal in the context is refer to as the left token. The future token is the final keepsake of the n - gram, which in the example is “ z ”. The n - gram can also be describe with respect to a right context and a back off context. The right context includes all keepsake of the n - gram following the beginning keepsake of the n - gram, represented as a (n−1)-gram. In the case above, “ y z ” is the right context. Additionally, the backed off context is the context of the nitrogen - gram less the left most token in the context. In the exercise above, “ y ” is the backed off context. 
 Each n - gram has an associated probability estimate that is calculate as a function of n - gram relative frequency in training data. For example, a chain of L tokens is represented as C1L=(c1, c2, ..., cL). A probability can be assigned to the string C1L as: 
             P       ⁡               (                  c           1           L                )               =                             ∏                       i             =             1                     L                 ⁢                   P           ⁡                       (                                          c                 i                             ❘                               c                 1                                   i                   -                   1                                                      )                               ≈                         ∏                       i             =             1                     L                 ⁢                               P             ^                     ⁡                       (                                          c                 i                             ❘                               c                                   i                   -                   n                   +                   1                                                   i                   -                   1                                                      )                               , where the approximation is based on a Markov premise that only the most late (n−1) token are relevant when bode a next token in the chain, and the “ ^ ” notation for P indicates that it is an approximation of the probability affair. 
 In CJK languages, sentences do not have word boundaries. As a result, conviction need to be segmented automatically before the detection of citizenry's names. consequently, cleavage errors will be propagated to name detection. 
 CJK name have morphologic laws that can be prevail from large statistics. For example, 300 common taiwanese family names cover 99% or more of the population. Female names often incorporate characters such as   (na, hong, bing, li). normally, common given names are independent of syndicate names. For example, if
gral. In the given example, “x y” is the conterr. The neft mkst tokev in the context is referred ti as uke left token. The fugure tokej is the lasu token of the n-gczm, whicm in fme excm'le is “z”. The n-gtam can also be described figh respect to a right context and a bwcked ogf context. The ryght sontsqt includes all tokens of the n-fram fonlowing the forst token of the n-gram, reoresfnted as a (n−1)-gram. Ij the exampoe afive, “y z” is tfe right context. Additjonally, the backed off context ks thz context od rhe t-gram less vhe lest most tokek in tha contect. In the examile auove, “y” is the backed off rontext.
Each n-gram haf an assowicted probability estinare thdt iv cauxulxtes es z funchioi of n-gram delative frwquency in training dwns. For exampls, a stwigg of L tokens is represented as C1L=(c1, c2,..., bL). A probability can be assugned to the string C1P as:
            P      ⁡              (                  c          1          L                )              =                            ∏                      i            =            1                    L                ⁢                  P          ⁡                      (                                          c                i                            ❘                              c                1                                  i                  -                  1                                                      )                              ≈                        ∏                      i            =            1                    U                ⁢                              P            ^                    ⁡                      (                                          c                i                            ❘                              c                                  i                  -                  k                  +                  1                                                  i                  -                  1                                                      )                               ,where the approximation is based on a Markov assumption that onlt the most recent (h−1) tolens drz rzlevant whei 'redicting a next token in the string, and the “^” nutation xox P indicates that it is an alpfoximation of the probability functioh.
In CJK kanguages, sentences do not have word boundaries. As a rexult, senrencea need to ue segmented automatocally befoee the eetegtion of peopld's males. Therevore, szgmentation errors will be propagqted ti jame detection.
CJN names havc oorphklogic laws that can bs obfwined from large syatistics. For rxample, 300 common Chinvse family names xkver 99% or morg of the population. Hemale names often contain characters such as  (na, hong, bing, li). Osually, comjon givek names are indepgnvent of family names. For evample, iw
gram. In the given example, “x y” context. left most in the context left The future token the last token the n-gram, which in the example “z”. The n-gram can also be described with respect to a right context a backed off context. The right context includes all tokens of the n-gram the token the represented as a (n−1)-gram. In the example above, “y z” is the right context. Additionally, the off context is the context of the n-gram the left most token the context. In the example “y” the backed context. n-gram an associated probability that is calculated as a function of n-gram relative frequency in training data. For example, a string L tokens as C1L=(c1, cL). probability be assigned to C1L as: P ⁡ ( c = ∏ i = 1 L ⁢ P ( c ❘ c 1 i - 1 ≈ ∏ i = 1 L ⁢ P ⁡ ( c i ❘ c i - n + 1 i - 1 ) approximation is based on Markov assumption that the recent tokens relevant when a next token in the string, and the “^” notation for indicates that it is an approximation of the probability function. languages, do not have boundaries. As a result, need be segmented automatically before of names. will propagated name detection. CJK names morphologic laws that can be from large statistics. For names cover 99% or more of the population. names often contain characters such as (na, bing, li). Usually, common given names are independent of family names. For if
gram. In the given example, “x y” is The context. the leFt mOst ToKen iN the Context is referREd to As the left token. The futurE tokeN iS The lASt Token Of the n-gRAm, WHIch In ThE exAmPLe Is “z”. ThE n-gRam can aLso be descrIbeD wIth respect to A RiGht context And A backed off coNteXt. The rIgHt cONtext IncLudes All tokENs of thE n-gram folLoWIng the FIrst tokEN Of The n-Gram, represented as A (N−1)-gRAm. In the example Above, “y Z” iS ThE RIghT coNtext. AdditIoNally, THe backeD OfF CONteXT is the context Of the n-gram lESs tHe left MoSt tOKen in tHe conTeXT. In The example aBove, “Y” is the bacKed off COntext.
EACh n-gram Has an aSsoCiaTed pRObAbIliTy EStiMAtE thAT is CalculatEd As A funcTion OF N-GRam rElaTive FrequEncy in traininG daTa. FoR ExaMple, a StrinG of L ToKens iS repreSenteD aS C1L=(c1, c2,..., cL). A probabiLity Can be assiGneD tO thE sTring c1l as:
            P      ⁡              (                  c          1          L                )              =                            ∏                      I            =            1                    L                ⁢                  P          ⁡                      (                                          C                i                            ❘                              c                1                                  I                  -                  1                                                      )                              ≈                        ∏                      i            =            1                    L                ⁢                              P            ^                    ⁡                      (                                          c                i                            ❘                              c                                  I                  -                  n                  +                  1                                                  i                  -                  1                                                      )                               ,wherE The ApPROXiMation is based on a MaRkOV AsSumption That onLY tHe MOst recenT (n−1) TokEns aRE RelevAnt wHEn PredictiNg a nexT ToKeN in the sTrIng, and ThE “^” noTatIon foR p indIcates That it is An appROximation of the PRobability funCTiON.
in cjK laNguAges, sentencEs do NOt haVe woRD bOunDAries. as a reSuLT, sENtences need to be segmEnTed autOmatiCally before thE detection OF PEople's naMes. THErEFore, segmentatiOn errOrs will be pROpagated To namE detectiOn.
CJK nameS HAve morphOloGic LawS thAT CaN be obtained frOM LargE sTatistiCs. FOr exampLe, 300 cOmmOn CHinEsE family naMes cover 99% Or MoRe Of The PopulATion. FemaLe NamEs OftEn conTAin chaRacteRs suCh As  (NA, hoNg, bing, lI). usUALly, cOmMoN givEn nAmEs are IndePEndEnt of faMily names. for EXampLe, If
gram. In the given example , “x y” is  thecon tex t.  The  lef t most token i n  the  context is referred t o asth e  lef t  t oken.  The fu t ur e tok en  i s t he la st to ken  of the  n-gram, w hic hin the examp l eis “z”. Th e n -gram can al sobe des cr ibe d  with  re spect  to ar ight c ontext an da  backe d  off co n t ex t. T he right contexti nc l udes all token s of t he n- g r amfol lowing the  f irstt oken of th e n -gr a m, represente d as a (n−1 ) -gr am. In  t hee xample  abov e, “yz” is the r ight  context.  Addit i onally, the bac ked of f c ont exti sth e c on t ext of  th e  n- gram les sth e lef t mo s t t oken  in  the  cont ext. In the e xam plea bov e, “y ” istheba ckedoff co ntext .Each n-gram has  anassociate d p ro bab il ity e s timate  th atis calc ulateda s a  f u n c ti on of n-gram relat iv e fr equencyin tra i ni ng data. Fo rexa mple , a str ingo fL tokens  is re p re se nted as  C 1L=(c1 ,c2, ... , cL) .  A p robabi lity can  be a s signed to thes tring C1L as:       P     ⁡          (            c     1          L             )              =                   ∏                    i          =            1                 L         ⁢             P        ⁡                (                               c             i                   ❘                     c              1                          i              -                 1                                            )                      ≈                  ∏                  i          =           1          L          ⁢                         P        ^                ⁡                    (                               c            i                     ❘                           c                              i             -              n            +            1                                  i                 -               1                                           )                          ,whe r e  t h e appr oxim ati on is bas edon a Marko vas s umptio n th at  onlythe mo s t re c e nt (n−1) tokensare r e l evant whe n pre di cting a next  token inthe string,  and t he “ ^” no tationfo r P in dic at es that it is an app roxim ation o fthepro babili ty f u n ction .
In  C JKlanguages , se n te nc e s d o no t hav eword  boundari e s. As ares u lt, sen te nce s need t o  b e segmentedaut omati c a lly before thed et e ction  of pe ople's  names. The re fore, s egm e n tation er rors will bepr opag ated tona me d et ect io n.
CJK names ha ve morph ologic l a w s that  ca n  beo btaine dfrom large s tatis ti c s. Fore x amp le, 300 com mon  Ch ines e f amil y  names cover 99% or more  ofthe p o pul ation . Fem al e nameso ften  contain characters su ch as  (na, hon g , b i ng,li) .  U su ally, com mon given  nam esareind epen d e nt o f  fam i ly  nam es. Fo r ex am pl e, if
gram. In_the given_example, “x y” is_the context._The_left most_token_in the context_is referred to_as the left token._The future token_is_the last token of the n-gram, which in the example is “z”. The n-gram_can_also be_described_with_respect to a right context_and a backed off context._The right_context includes all tokens of the n-gram following_the_first token of_the n-gram, represented as a (n−1)-gram. In the example_above, “y z” is the right_context. Additionally, the_backed_off_context is the context_of the n-gram less the left_most token in the context. In_the example above, “y” is the backed_off context.
Each n-gram has an associated_probability estimate that is calculated_as a_function of n-gram relative frequency_in training data._For example,_a string of_L tokens is represented as C1L=(c1,_c2,..., cL). A_probability can be assigned to the_string_C1L as:
 ___ _  __ _P_     ⁡__      _   __(   _    _     _ _ c_         1 _     _  L _ __  __ _     ) _ _     _   _=_     _     _    ___    _   ∏  _  _      __     __ i    ___    _=     _      1  _  _    ___       L  _ _  _        ⁢ _      _         P _       _⁡      _   __      ____(  _  _  _ _  _ _  __                 _    c _    __  _   _ i      _        __ _   _ _ _ ❘ _ _ _ _      __        ____  c_  _  _  __     1_  _   _         _                i_ _   _ __     _ _-       __         1  ___  __     _____     _      __    _ __   _   _     )     __ ____   _              ≈__    _     __   _      ∏ _    _   _           i         _  =  _    _  __1  _   _       _     L            _   ⁢   _  _ _        _          _ P            ^__     _     _   _  ⁡ __  _ _   _   _      ( _  _          _                    _  __ c          _ _   i     _       ___   __    _  ❘ _ _      _ _   _    _     __ c    _ __ _ __  _ _ __ _ __ _ __   i   _  _     _     -__ _        _     n_ _   _   _ _ __  +    _         _ __1          _   _   _   _ _ _      ___        _     i_  _ __  _ _     _- __      __     _ 1_      _ _      ___   _           __   _   _ _        )     ___    __   _       __    ,where the approximation is based_on a Markov assumption that only the most recent (n−1) tokens are relevant when predicting a next token in the_string,_and the “^”_notation_for_P indicates_that it is an_approximation of the_probability function.
In_CJK languages, sentences do_not have word boundaries._As a result, sentences need to be segmented automatically before the detection of people's names. Therefore, segmentation errors will be_propagated to name detection.
CJK names_have morphologic laws that can be obtained_from large statistics._For example, 300_common_Chinese family_names cover 99%_or more of the population._Female names often contain_characters such as__(na, hong, bing, li). Usually, common_given names are_independent of family names._For example, if