title: How the α value is calculated in katz backoff language model?
body: <p>Im currently working on the implementation for katz backoff smoothing language model. i have some confusion about the recursive backoff and α calculation lower order models. Suppose the katz model for trigram is given as </p>
  
  <p>P<sub>katz</sub>(w<sub>i</sub>|w<sub>i-2</sub>w<sub>i-1</sub>)= P(w<sub>i</sub>|w<sub>i-2</sub>w<sub>i-1</sub>), if C(w<sub>i-2</sub>w<sub>i-1</sub>w<sub>i</sub>) > 0 <br/> &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; or  α<sub>1</sub>P(w<sub>i</sub>|w<sub>i-1</sub>), if C(w<sub>i-2</sub>w<sub>i-1</sub>w<sub>i</sub>) = 0 and C(w<sub>i-1</sub>w<sub>i</sub>) > 0<br/> &nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp;&nbsp; &nbsp; &nbsp; &nbsp;or  α<sub>2</sub>P(w<sub>i</sub>), otherwise</p>
  
  <p>I also know that α1 for bigram case is calculated from the discounts found from trigram model but my confusion is how it calculates the α2 for lower order unigram. Does it use the discounts from the trigram model or it uses discounts from bigram model if both trigram and bigram case has zero evidence? </p>