I need a neat data structure suggestion to store a very large dataset (to train Naive Bayes in Python)
If you assume you didn't care about multiple occurrences of each word in an email, then all you really need to know is (that is, your features are booleans):
For each feature, what is the count of positive associations and negative associations?
You can do this online very easily in one pass, keeping track of just those two numbers for each feature.
The non-boolean features means you'll have to discretize the features some how, but you aren't really asking about how to do that.
I am going to implement Naive Bayes classifier with Python and classify e-mails as Spam or Not spam. I have a very sparse and long dataset with many entries. Each entry is like the following:
1 9:3 94:1 109:1 163:1 405:1 406:1 415:2 416:1 435:3 436:3 437:4 ...
Where 1 is label (spam, not spam), and each pair corresponds to a word and its frequency. E.g. 9:3 corresponds to the word 9 and it occurs 3 times in this e-mail sample.
I need to read this dataset and store it in a structure. Since it's a very big and sparse dataset, I'm looking for a neat data structure to store the following variables:

the index of each e-mail
label of it (1 or -1)
word and it's frequency per each e-mail
I also need to create a corpus of all words and their frequency with the label information

Any suggestions for such a data structure?