1. A joint decoding apparatus that joins a tokenization process and a translation process, the joint decoding apparatus comprising: a candidate token generation unit configured to generate a plurality of candidate tokens by applying a maximum entropy model to an input character sequence; a probability calculation unit configured to calculate a tokenization probability of each of the generated candidate tokens by using a language model; an out of vocabulary (OOV) processing unit configured to process an OOV by using a word count and discount information for the OOV among the generated candidate tokens; and a translation unit configured to generate a corresponding phrase corresponding to the input character sequence according to a translation model by using calculated probability values of the candidate tokens and an OOV processing result.
1. A joint decoding apparatus that joins a tokenization process and a translation process, the joint decoding apparatus comprising: a candidate token generation unit configured to generate a plurality of candidate tokens by applying a maximum entropy model to an input character sequence; a probability calculation unit configured to calculate a tokenization probability of each of the generated candidate tokens by using a language model; an out of vocabulary (OOV) processing unit configured to process an OOV by using a word count and discount information for the OOV among the generated candidate tokens; and a translation unit configured to generate a corresponding phrase corresponding to the input character sequence according to a translation model by using calculated probability values of the candidate tokens and an OOV processing result. 2. The joint decoding apparatus of claim 1 , further comprising: a translation model database configured to store a translation model trained from tokenized data of a parallel corpus; and a language model database configured to store a language model trained from a monolingual corpus.