docstr: Chinese G2P module. This module first converts Chinese characters into pinyin sequences using pypinyin, then
      pinyin sequences would be further converted into phoneme sequences by looking them up in the `phoneme_dict`.
      This G2P module also works with Chinese/English bilingual sentences where English words would be converted
      into letters. It is advised to attach prefix symbols for Chinese phonemes and tones to discriminate them
      from English letters to avoid any potential symbol set overlaps.
  Args:
      phoneme_dict (str, Path, Dict): Path to pinyin_dict_nv_22.10.txt dict file or a dict object.
      phoneme_prefix (str): Prepend a special symbol to any phonemes in order to distinguish phonemes from
          graphemes because there may be overlaps between the two sets. Phoneme dictionary typically applies
          uppercase initials and finals. It is suggested to choose a prefix that
          is not used or preserved somewhere else. Default to "#".
      phoneme_case (str): Specify the case chosen from `"lower"`, `"upper"`, or `"mixed"`, and process the
          cases of Chinese phonemes. Default to `"upper"`.
      tone_prefix (str): Prepend a special symbol to any tone digits. Default to "#".
      ascii_letter_prefix (str): Prepend a special symbol to any ASCII letters. Default to "".
      ascii_letter_case (str): Specify the case chosen from `"lower"`, `"upper"`, or `"mixed"`, and process the
          cases of non-Chinese words. Default to `"lower"`.
      word_tokenize_func: Function for tokenizing text to words.
          It has to return List[Tuple[Union[str, List[str]], bool]] where every tuple denotes word representation
              and flag whether to leave unchanged or not.
          It is expected that unchangeable word representation will be represented as List[str], other cases are
              represented as str.
          It is useful to mark word as unchangeable which is already in phoneme representation.
      apply_to_oov_word: Function that will be applied to out of phoneme_dict word.
      word_segmenter: method that will be applied to segment utterances into words for better polyphone disambiguation.
docstr_tok: ["Chinese","G2P","module",".","This","module","first","converts","Chinese","characters","into","pinyin","sequences","using","pypinyin",",","then","pinyin","sequences","would","be","further","converted","into","phoneme","sequences","by","looking","them","up","in","the","`","phoneme_dict","`",".","This","G2P","module","also","works","with","Chinese\/English","bilingual","sentences","where","English","words","would","be","converted","into","letters",".","It","is","advised","to","attach","prefix","symbols","for","Chinese","phonemes","and","tones","to","discriminate","them","from","English","letters","to","avoid","any","potential","symbol","set","overlaps",".","Args",":","phoneme_dict","(","str",",","Path",",","Dict",")",":","Path","to","pinyin_dict_nv_22.10.txt","dict","file","or","a","dict","object",".","phoneme_prefix","(","str",")",":","Prepend","a","special","symbol","to","any","phonemes","in","order","to","distinguish","phonemes","from","graphemes","because","there","may","be","overlaps","between","the","two","sets",".","Phoneme","dictionary","typically","applies","uppercase","initials","and","finals",".","It","is","suggested","to","choose","a","prefix","that","is","not","used","or","preserved","somewhere","else",".","Default","to","``","#","''",".","phoneme_case","(","str",")",":","Specify","the","case","chosen","from","`","``","lower","''","`",",","`","``","upper","''","`",",","or","`","``","mixed","''","`",",","and","process","the","cases","of","Chinese","phonemes",".","Default","to","`","``","upper","''","`",".","tone_prefix","(","str",")",":","Prepend","a","special","symbol","to","any","tone","digits",".","Default","to","``","#","''",".","ascii_letter_prefix","(","str",")",":","Prepend","a","special","symbol","to","any","ASCII","letters",".","Default","to","``","''",".","ascii_letter_case","(","str",")",":","Specify","the","case","chosen","from","`","``","lower","''","`",",","`","``","upper","''","`",",","or","`","``","mixed","''","`",",","and","process","the","cases","of","non-Chinese","words",".","Default","to","`","``","lower","''","`",".","word_tokenize_func",":","Function","for","tokenizing","text","to","words",".","It","has","to","return","List","[","Tuple","[","Union","[","str",",","List","[","str","]","]",",","bool","]","]","where","every","tuple","denotes","word","representation","and","flag","whether","to","leave","unchanged","or","not",".","It","is","expected","that","unchangeable","word","representation","will","be","represented","as","List","[","str","]",",","other","cases","are","represented","as","str",".","It","is","useful","to","mark","word","as","unchangeable","which","is","already","in","phoneme","representation",".","apply_to_oov_word",":","Function","that","will","be","applied","to","out","of","phoneme_dict","word",".","word_segmenter",":","method","that","will","be","applied","to","segment","utterances","into","words","for","better","polyphone","disambiguation","."]
code: def __init__(
      self,
      phoneme_dict: Union[str, pathlib.Path, Dict[str, List[str]]],
      phoneme_prefix: str = "#",
      phoneme_case: str = "upper",
      tone_prefix: str = "#",
      ascii_letter_prefix: str = "",
      ascii_letter_case: str = "lower",
      word_tokenize_func=None,
      apply_to_oov_word=None,
      mapping_file: Optional[str] = None,
      word_segmenter: Optional[str] = None,
  ):
      """
      Chinese G2P module. This module first converts Chinese characters into pinyin sequences using pypinyin, then
          pinyin sequences would be further converted into phoneme sequences by looking them up in the `phoneme_dict`.
          This G2P module also works with Chinese/English bilingual sentences where English words would be converted
          into letters. It is advised to attach prefix symbols for Chinese phonemes and tones to discriminate them
          from English letters to avoid any potential symbol set overlaps.
      Args:
          phoneme_dict (str, Path, Dict): Path to pinyin_dict_nv_22.10.txt dict file or a dict object.
          phoneme_prefix (str): Prepend a special symbol to any phonemes in order to distinguish phonemes from
              graphemes because there may be overlaps between the two sets. Phoneme dictionary typically applies
              uppercase initials and finals. It is suggested to choose a prefix that
              is not used or preserved somewhere else. Default to "#".
          phoneme_case (str): Specify the case chosen from `"lower"`, `"upper"`, or `"mixed"`, and process the
              cases of Chinese phonemes. Default to `"upper"`.
          tone_prefix (str): Prepend a special symbol to any tone digits. Default to "#".
          ascii_letter_prefix (str): Prepend a special symbol to any ASCII letters. Default to "".
          ascii_letter_case (str): Specify the case chosen from `"lower"`, `"upper"`, or `"mixed"`, and process the
              cases of non-Chinese words. Default to `"lower"`.
          word_tokenize_func: Function for tokenizing text to words.
              It has to return List[Tuple[Union[str, List[str]], bool]] where every tuple denotes word representation
                  and flag whether to leave unchanged or not.
              It is expected that unchangeable word representation will be represented as List[str], other cases are
                  represented as str.
              It is useful to mark word as unchangeable which is already in phoneme representation.
          apply_to_oov_word: Function that will be applied to out of phoneme_dict word.
          word_segmenter: method that will be applied to segment utterances into words for better polyphone disambiguation.
      """
      assert (
          phoneme_dict is not None
      ), "Please set the phoneme_dict path."
      assert word_segmenter in [
          None,
          "jieba",
          "pkuseg",
      ], f"{word_segmenter} is not supported now. Please choose correct word_segmenter."
  
      if phoneme_prefix is None:
          phoneme_prefix = ""
      if tone_prefix is None:
          tone_prefix = ""
      if ascii_letter_prefix is None:
          ascii_letter_prefix = ""
  
      # phonemes
      phoneme_dict = (
          self._parse_as_pinyin_dict(
              phoneme_dict, phoneme_prefix, phoneme_case
          )
          if isinstance(phoneme_dict, str)
          or isinstance(phoneme_dict, pathlib.Path)
          else phoneme_dict
      )
      self.phoneme_list = sorted(
          {pron for prons in phoneme_dict.values() for pron in prons}
      )
  
      # tones
      self.tone_dict = {
          str(x): tone_prefix + str(x) for x in range(1, 6)
      }
      self.tone_list = sorted(self.tone_dict.values())
  
      # ascii letters
      self.ascii_letter_dict = {
          x: ascii_letter_prefix + x
          for x in get_grapheme_character_set(
              locale="en-US", case=ascii_letter_case
          )
      }
      self.ascii_letter_list = sorted(self.ascii_letter_dict)
      self.ascii_letter_case = ascii_letter_case
  
      if apply_to_oov_word is None:
          logging.warning(
              "apply_to_oov_word=None, This means that some of words will remain unchanged "
              "if they are not handled by any of the rules in self.parse_one_word(). "
              "This may be intended if phonemes and chars are both valid inputs, otherwise, "
              "you may see unexpected deletions in your input."
          )
  
      super().__init__(
          phoneme_dict=phoneme_dict,
          word_tokenize_func=word_tokenize_func,
          apply_to_oov_word=apply_to_oov_word,
          mapping_file=mapping_file,
      )
  
      if word_segmenter == "jieba":
          try:
              import jieba
          except ImportError as e:
              logging.error(e)
  
          # Cut sentences into words to improve polyphone disambiguation
          self.word_segmenter = jieba.cut
      elif word_segmenter == "pkuseg":
          try:
              import pkuseg
          except ImportError as e:
              logging.error(
                  "`pkuseg` is not the default word segmenter for Chinese in NeMo. Please install it manually by running `pip install pkuseg`, or choose `jieba` segmenter instead."
              )
  
          self.word_segmenter = pkuseg.pkuseg().cut
      else:
          self.word_segmenter = lambda x: [x]
  
      try:
          from pypinyin import Style, lazy_pinyin
          from pypinyin_dict.pinyin_data import cc_cedict
      except ImportError as e:
          logging.error(e)
  
      # replace pypinyin default dict with cc_cedict.txt for polyphone disambiguation
      cc_cedict.load()
  
      self._lazy_pinyin = lazy_pinyin
      self._Style = Style
  
file_path: NeMo/nemo/collections/tts/g2p/models/zh_cn_pinyin.py
imports: import pathlib
  from collections import defaultdict
  from typing import Dict, List, Optional, Union
  from nemo.collections.common.tokenizers.text_to_speech.ipa_lexicon import get_grapheme_character_set
  from nemo.collections.tts.g2p.models.base import BaseG2p
  from nemo.collections.tts.g2p.utils import set_grapheme_case
  from nemo.utils import logging
label_desc: Use image node_id 1 to create a new ChineseG2p object from inherited base classes: BaseG2p with example: obj = ChineseG2p(phoneme_dict, phoneme_prefix, phoneme_case, tone_prefix, ascii_letter_prefix, ascii_letter_case, word_tokenize_func, apply_to_oov_word, mapping_file, word_segmenter)