NLP: What are some popular packages for multi-word tokenization?
The multiword tokenizer 'nltk.tokenize.mwe' basically merges a string already divided into tokens, based on a lexicon, from what I understood from the API documentation.

One thing you can do is tokenize and tag all words with it's associated part-of-speech (PoS) tag, and then define regular expressions based on the PoS-tags to extract interesting key-phrases. 

For instance, an example adapted from the NLTK Book Chapter 7 and  this blog post:

def extract_phrases(my_tree, phrase):
   my_phrases = []
   if my_tree.label() == phrase:
      my_phrases.append(my_tree.copy(True))

   for child in my_tree:
       if type(child) is nltk.Tree:
            list_of_phrases = extract_phrases(child, phrase)
            if len(list_of_phrases) &gt; 0:
                my_phrases.extend(list_of_phrases)

    return my_phrases



def main():
    sentences = ["The little yellow dog barked at the cat",
                 "He studies Information Technology"]

    grammar = "NP: {&lt;DT&gt;?&lt;JJ&gt;*&lt;NN&gt;|&lt;NNP&gt;*}"
    cp = nltk.RegexpParser(grammar)

    for x in sentences:
        sentence = pos_tag(tokenize.word_tokenize(x))
        tree = cp.parse(sentence)
        print "\nNoun phrases:"
        list_of_noun_phrases = extract_phrases(tree, 'NP')
        for phrase in list_of_noun_phrases:
            print phrase, "_".join([x[0] for x in phrase.leaves()])


You defined a grammar based on regex over PoS-tags:

grammar = "NP: {&lt;DT&gt;?&lt;JJ&gt;*&lt;NN&gt;|&lt;NNP&gt;*}"
cp = nltk.RegexpParser(grammar)


Then you applied it to a tokenized and tagged sentence, generating a Tree:

sentence = pos_tag(tokenize.word_tokenize(x))
tree = cp.parse(sentence)


Then you use extract_phrases(my_tree, phrase) to recursively parse the Tree and extract sub-trees labeled as NP. The example above would extract the following noun-phrases:

Noun phrases:
(NP The/DT little/JJ yellow/JJ dog/NN) The_little_yellow_dog
(NP the/DT cat/NN) the_cat

Noun phrases:
(NP Information/NNP Technology/NNP) Information_Technology


There is a great blog post by Burton DeWilde about many more ways to extract interesting keyphrases: Intro to Automatic Keyphrase Extraction