Too many values to unpack using NLTK and Pandas in Python
Question: I am trying out different things to make the NLTK's naive bayes work using the
NLTK and Pandas modules, but I am getting the "too many values to unpack"
error.

    
    
    import pandas as pd
    from pandas import DataFrame, Series
    import numpy as np
    import re
    import nltk
    
    ### Remove cases with missing name or missing ethnicity information
    def read_file():
        data = pd.read_csv("C:\sample.csv", encoding="utf-8")
        frame = DataFrame(data)
        frame.columns = ["Name", "Gender"]
    
        return frame
    #read_file()
    
    def gender_features(word):
        return {'last_letter': word[-1]}
    #gender_features()
    
    frame = read_file()
    featuresets = [(gender_features(n), gender) for (n, gender) in frame]
    train_set, test_set = features[500:], featuresets[:500]
    classifier = nltkNaiveBayesClassifier.train(train_set)
    


Answer: I suspect you are trying to do something bigger than name classification when
using `panadas.DataFrame` because the `DataFrame` object is normally used when
you have limited RAM and wants to makes use of diskspace as you iterate
through the data to extract features:

> a 2-dimensional labeled data structure with columns of potentially different
> types. You can think of it like a spreadsheet or SQL table, or a dict of
> Series objects. It is generally the most commonly used pandas object. Like
> Series, DataFrame accepts many different kinds of input:
>
>   * Dict of 1D ndarrays, lists, dicts, or Series
>   * 2-D numpy.ndarray
>   * Structured or record ndarray
>   * A Series
>   * Another DataFrame
>

I suggest you go through the `pandas` tutorial to learn about the library
first: <http://pandas.pydata.org/pandas-docs/dev/tutorials.html>

And then learn about the NLTK classification from
<http://www.nltk.org/book/ch06.html>

* * *

Firstly, there are several things wrong in how you access `pandas.DataFrame`
object.

To iterate through the rows of the dataframe, you should do this:

    
    
    # Read file into pandas dataframe
    df = DataFrame(pd.read_csv('sample.csv'))
    df.columns = ['name', 'gender']
    
    for index, row in df.iterrows():
        print row['name'], row['gender']
    

Next to train a classifier, you should do this:

    
    
    import numpy as np
    import pandas as pd
    from pandas import DataFrame, Series
    
    from nltk.corpus import names
    from nltk.classify import NaiveBayesClassifier as nbc
    
    # Create a sample.csv file
    male_names = [','.join([i,'m']) for i in names.words('male.txt')]
    female_names = [','.join([i,'m']) for i in names.words('female.txt')]
    with open('sample.csv', 'w') as fout:
        fout.write('\n'.join(male_names+female_names))
    
    # Feature extractor function.
    def gender_features(word):
        return {'last_letter': word[-1]}
    
    # Read file into pandas dataframe
    df = DataFrame(pd.read_csv('sample.csv'))
    df.columns = ['name', 'gender']
    
    # Extract features. 
    featuresets = [(gender_features(name), gender) for index, (name, gender) in df.iterrows()]
    # Split train and test set
    train_set, test_set = featuresets[500:], featuresets[:500]
    # Train a classifier
    classifier = nbc.train(train_set)
    # Test classifier on "Neo"
    print classifier.classify(gender_features('Neo'))
    

[out]:

    
    
    m
    

