By means of parts of speech tagging, words of a given sentence can be assumed to be noun/verb etc, but if the sentence is for instance:My favourite book is harry potter and the prizoner of azkabannote that the inputs I receive would be from a chat interface so having a fixed format for the data can't be expected. Is there a way to identify harry potter and the prizoner of azkaban as a proper noun from such messages?Currently this query tags as:My|PRP$ favourite|JJ book|NN is|VBZ harry|JJ potter|NN and|CC the|DT prizoner|NN of|IN azkaban|NNI would like to know if this can be handled some way, or if there is another algorithm that can handle this?
Tagging parts of speech when proper noun is a composite
nlp
I guess your problem is a form of NER (Named-Entity Recognition) tagging. NER tags consist of PER(person), LOC(location), ORG(organization) and MISC(miscellaneous) and O(other). with the help of a NER-tagger algorithm, you probably would have:My(O) favorite(O) book(O) is(O) harry(B-MISC) potter(I-MISC) and(I-MISC) the(I-MISC) prisoner(I-MISC) of(I-MISC) azkaban(I-MISC).now you can tokenize your text with identifying different NER tags, and join B- and I- prefixes with the same NER tag. e.g. you have harry potter and ... as a single token, which is a MISC and starts from harry (since harry is B-MISC) and ends to azkaban (since azkaban has the last consecutive I-MISC tag). Now you can let your POS tagger act the harry potter ... as a single token, and it must tag it with NNAnother solution for this problem is called chunking. It works based on a set of rules and detect Noun Phrases (NPs). Here, you define a rule and create the regexp statement for it. e.g. you define all consecutive NNs as a single NP (harry potter): <NN>+ or a DT followed by a string of NNs (the brown fox): <DT>?<NN>+. Now you tokenize your text based on a regexp matching method. but in your case, chunking such a long NP is almost impossible. because a rule that finds this NP as a NP candidate, would by mistake find a lot of other string of words as a NP candidate too, while they're not a NP at all. (see nltk: information extraction)