: : - : ggolden @mac com : : - : tpamsler@ucdavis edu : : - : lance@indiana edu : : - : vrajgopalan@ucmerced edu the gmodel py program handles number of data cleaning tasks domain names are truncated to two levels for comorgeduand net other domain names are truncated to three levels so si umich edu becomes umich edu and caret cam ac uk becomes cam ac uk email addresses are also forced to lower caseand some of the @gmane org address like the following arwhyte- axycvo tyhxe+lvdladg@public gmane org are converted to the real address whenever there is matching real email address elsewhere in the message corpus in the mapping sqlite database there are two tables that allow you to map both domain names and individual email addresses that change over the lifetime of the email list for examplesteve githens used the following email addresses as he changed jobs over the life of the sakai developer lists-githens@northwestern edu sgithens@cam ac uk swgithen@mtu edu we can add two entries to the mapping table in mapping sqlite so gmodel py will map all three to one addresss-githens@northwestern edu -swgithen@mtu edu sgithens@cam ac uk -swgithen@mtu edu you can also make similar entries in the dnsmapping table if there are multiple dns names you want mapped to single dns the following mapping was added to the sakai dataiupui edu -indiana edu so all the accounts from the various indiana university campuses are tracked together you can rerun the gmodel py over and over as you look at the dataand add mappings to make the data cleaner and cleaner when you are doneyou will have nicely indexed version of the email in index sqlite this is the file to use to do data analysis with this filedata analysis will be really quick the firstsimplest data analysis is to determine "who sent the most mail?and "which organization sent the most mail"this is done using gbasic py