title: Named Entity recognition with openNLP (default model)
body: <p>Can anyone point out the algorithm(s) used by openNLP NameFinder module?
  The code is complex and only sparsely documented and playing with it as a black box (with the default model provided) gives me the impression that it is mostly heuristic. 
  Here are some examples for input and output:</p>
  
  <p>Input:</p>
  
  <blockquote>
    <p>John Smith is frustrated. </p>
    
    <p>john smith is frustrated.</p>
    
    <p>Barak Obama is frustrated.</p>
    
    <p>Hugo Chavez is frustrated. (no more)</p>
    
    <p>Jeff Atwood is frustrated.</p>
    
    <p>Bing Liu is frustrated with openNLP NER module.</p>
    
    <p>Noam Chomsky is frustrated with the world.</p>
    
    <p>Jayden Smith is frustrated.</p>
    
    <p>Smith Jayden is frustrated.</p>
    
    <p>Lady Gaga is frustrated. </p>
    
    <p>Ms. Gaga is frustrated. </p>
    
    <p>Mrs. Gaga is frustrated. </p>
    
    <p>Jayden is frustrated.</p>
    
    <p>Mr. Liu is frustrated.</p>
  </blockquote>
  
  <p>Output (I changed diamonds to square brackets) :</p>
  
  <blockquote>
    <p>[START:person] John Smith [END] is frustrated.</p>
    
    <p>john smith is frustrated.</p>
    
    <p>[START:person] Barak Obama [END] is frustrated.</p>
    
    <p>Hugo Chavez is frustrated. (no more)</p>
    
    <p>[START:person] Jeff Atwood [END] is frustrated.</p>
    
    <p>Bing Liu is frustrated with openNLP NER module.</p>
    
    <p>[START:person] Noam Chomsky [END] is frustrated with the world.</p>
    
    <p>Jayden [START:person] Smith [END] is frustrated.</p>
    
    <p>[START:person] Smith [END] [START:person] Jayden [END] is frustrated.</p>
    
    <p>Lady Gaga is frustrated.</p>
    
    <p>Ms. Gaga is frustrated.</p>
    
    <p>Mrs. Gaga is frustrated.</p>
    
    <p>Jayden is frustrated.</p>
    
    <p>Mr. Liu is frustrated.</p>
  </blockquote>
  
  <p>It seems that the model simply learns a fixed list of names that are annotated in the training data and allows some tiling and combinations. 
  Two notable (FN) examples are:</p>
  
  <ol>
  <li>Strong name indicators such as Mr. and Mrs. are ignored.  </li>
  <li>Jayden (#4 most popular name in the US in 2011) wasn't identified while the following 'Smith' (in "Jayden Smith...") was identified. I suspect that the model "thinks" that the capitalized Jayden in the beginning of the sentence is due the beginning of sentence and not due being a NE. Reversing the order, "Smith Jayden" as a hint (assuming 1), openNLP identifies it as two distinctive NEs, unlike other full names such as "John Smith", maybe suggesting that 'Smith' is in the last-names list...</li>
  </ol>
  
  <p>-> I'm puzzled and frustrated and if anyone could point me to the algorithm (or verify it sucks) I'll be thankful. </p>
  
  <p>p.s. both Stanford and UIUC NER systems perform much better with some subtle differences that are interesting but off topic (this question is too long as is)</p>