title: Naive Bayes, not so Naive?
body: <p>I have a Naive Bayes classifier (implemented with WEKA) that looks for uppercase letters.</p>
  
  <pre><code>contains_A
  contains_B
  ...
  contains_Z
  </code></pre>
  
  <p>For a certain class the word LCD appears in almost every instance of the training data. When I get the probability for "LCD" to belong to that class it is something like 0.988. win.</p>
  
  <p>When I get the probability for "L" I get a plain 0 and for "LC" I get 0.002. Since features are naive, shouldn't the L, C and D contribute to overall probability independently, and as a result "L" have some probability, "LC" some more and "LCD" even more?</p>
  
  <p>At the same time, the same experiment with an MLP, instead of having the above behavior it gives percentages of 0.006, 0.5 and 0.8</p>
  
  <p>So the MLP does what I would expect a Naive Bayes to do, and vise versa. Am I missing something, can anyone explain these results?</p>