4.7 - ACE/Analysis/sampling_over_time.py
def multiple_run_sampling(group_names, all_groups_contribs, all_groups_toks, n_runs=5,
                          snap_samples_per_user=10, snap_num_users=100,
                          test_sample_size=1000, replace=False, n_words_per_contrib=60,
                          win_func=get_time_windows, win_size=360, win_step=90,
                          comparison_method="CE"):
    """Runs the sampling multiple times and calculates a comparison at each run.

    Performs multiple runs of sampling and calculation of comparison metric.
    For each run, creates a "snapshot" sample of 1000 contributions to train a snapshot model on.
    Also creates a "test" sample of 1000 contributions for which to calculate a comparison, e.g. cross-entropy or KL-Divergence.
    Then calculates the cross entropy of each contribution in the test sample w.r.t the snapshot models.
    It does this calculation for every combination of groups.

    :param group_names: a list of the groups in the given data.
    :param all_group_contribs: a list of Dataframes of contributions - one for each group.
    :param all_groups_toks: a list that matches "all_group_contribs" but consists of a pandas Series of
                            tokenised contributions - one for each group.
    :param n_runs: the number of times to resample and recalculated cross entropy.
    :param comparison_func: the method used to compare the groups - must be "CE", "KLD", "CE_Fluct", or "KLD_Fluct".

    :returns: a list of dictionaries, one for each run. Each dictionary has another dictionary for each
                group which contains cross entropy results dataframes. One can find the information of a
                run as follows: "runs[group1][group2]" would contain a DataFrame with the cross entropies
                of test samples from group2 according to the snapshot model trained on group1.
    """

    comparison_methods = {
                            "CE": calculate_CE_per_group,
                            "KLD": calculate_KLD_per_group,
                            "CE_Fluct": calculate_CE_fluct_per_group,
                            "KLD_Fluct": calculate_KLD_fluct_per_group
                         }

    # Initialise empty output array.
    all_runs = []

    # Loop for each run.
    for run_num in range(n_runs):
        # Sample snapshot and test samples for each group.
        samples, test_contribs = get_samples_and_test_set(group_names,
                                                          all_groups_contribs,
                                                          all_groups_toks,
                                                          snap_samples_per_user=snap_samples_per_user,
                                                          snap_num_users=snap_num_users,
                                                          test_sample_size=test_sample_size,
                                                          replace=replace,
                                                          window_func=win_func,
                                                          window_size=win_size,
                                                          window_step=win_step)

        # Get all of the test tokens
        test_toks = {group_names[i]: all_groups_toks[i][test_contribs[group_names[i]].index] for i in range(len(group_names))}

        # Calculate the comparison values for all groups
        all_groups_comparison = comparison_methods[comparison_method](samples, test_contribs, test_toks, group_names, all_groups_toks, n_words_per_contrib)

        # Add dictionary for current run to output list.
        all_runs.append(all_groups_comparison)

    return all_runs
Runs the sampling multiple times and calculates a comparison at each run.

    Performs multiple runs of sampling and calculation of comparison metric.
    For each run, creates a "snapshot" sample of 1000 contributions to train a snapshot model on.
    Also creates a "test" sample of 1000 contributions for which to calculate a comparison, e.g. cross-entropy or KL-Divergence.
    Then calculates the cross entropy of each contribution in the test sample w.r.t the snapshot models.
    It does this calculation for every combination of groups.

    :param group_names: a list of the groups in the given data.
    :param all_group_contribs: a list of Dataframes of contributions - one for each group.
    :param all_groups_toks: a list that matches "all_group_contribs" but consists of a pandas Series of
                            tokenised contributions - one for each group.
    :param n_runs: the number of times to resample and recalculated cross entropy.
    :param comparison_func: the method used to compare the groups - must be "CE", "KLD", "CE_Fluct", or "KLD_Fluct".

    :returns: a list of dictionaries, one for each run. Each dictionary has another dictionary for each
                group which contains cross entropy results dataframes. One can find the information of a
                run as follows: "runs[group1][group2]" would contain a DataFrame with the cross entropies
                of test samples from group2 according to the snapshot model trained on group1.
    
Runs the sampling multiple times and calculates a comparison at each run.
Performs multiple runs of sampling and calculation of comparison metric.
For each run, creates a "snapshot" sample of 1000 contributions to train a snapshot model on.
Also creates a "test" sample of 1000 contributions for which to calculate a comparison, e.g. cross-entropy or KL-Divergence.
Then calculates the cross entropy of each contribution in the test sample w.r.t the snapshot models.
It does this calculation for every combination of groups.
Runs the sampling multiple times and calculates a comparison at each run . Performs multiple runs of sampling and calculation of comparison metric . For each run creates a " snapshot " sample of 1000 contributions to train a snapshot model on . Also creates a " test " sample of 1000 contributions for which to calculate a comparison e . g . cross - entropy or KL - Divergence . Then calculates the cross entropy of each contribution in the test sample w . r . t the snapshot models . It does this calculation for every combination of groups .
def multiple_run_sampling(group_names, all_groups_contribs, all_groups_toks, n_runs=5,
                          snap_samples_per_user=10, snap_num_users=100,
                          test_sample_size=1000, replace=False, n_words_per_contrib=60,
                          win_func=get_time_windows, win_size=360, win_step=90,
                          comparison_method="CE"):
    comparison_methods = {
                            "CE": calculate_CE_per_group,
                            "KLD": calculate_KLD_per_group,
                            "CE_Fluct": calculate_CE_fluct_per_group,
                            "KLD_Fluct": calculate_KLD_fluct_per_group
                         }
    all_runs = []
    for run_num in range(n_runs):
        samples, test_contribs = get_samples_and_test_set(group_names,
                                                          all_groups_contribs,
                                                          all_groups_toks,
                                                          snap_samples_per_user=snap_samples_per_user,
                                                          snap_num_users=snap_num_users,
                                                          test_sample_size=test_sample_size,
                                                          replace=replace,
                                                          window_func=win_func,
                                                          window_size=win_size,
                                                          window_step=win_step)
        test_toks = {group_names[i]: all_groups_toks[i][test_contribs[group_names[i]].index] for i in range(len(group_names))}
        all_groups_comparison = comparison_methods[comparison_method](samples, test_contribs, test_toks, group_names, all_groups_toks, n_words_per_contrib)
        all_runs.append(all_groups_comparison)
    return all_runs
def multiple_run_sampling ( group_names , all_groups_contribs , all_groups_toks , n_runs = 5 , snap_samples_per_user = 10 , snap_num_users = 100 , test_sample_size = 1000 , replace = False , n_words_per_contrib = 60 , win_func = get_time_windows , win_size = 360 , win_step = 90 , comparison_method = "CE" ) : comparison_methods = { "CE" : calculate_CE_per_group , "KLD" : calculate_KLD_per_group , "CE_Fluct" : calculate_CE_fluct_per_group , "KLD_Fluct" : calculate_KLD_fluct_per_group } all_runs = [ ] for run_num in range ( n_runs ) : samples , test_contribs = get_samples_and_test_set ( group_names , all_groups_contribs , all_groups_toks , snap_samples_per_user = snap_samples_per_user , snap_num_users = snap_num_users , test_sample_size = test_sample_size , replace = replace , window_func = win_func , window_size = win_size , window_step = win_step ) test_toks = { group_names [ i ] : all_groups_toks [ i ] [ test_contribs [ group_names [ i ] ] . index ] for i in range ( len ( group_names ) ) } all_groups_comparison = comparison_methods [ comparison_method ] ( samples , test_contribs , test_toks , group_names , all_groups_toks , n_words_per_contrib ) all_runs . append ( all_groups_comparison ) return all_runs
Runs the sampling multiple times and calculates a comparison at each run.
Runs the sampling multiple times and calculates a comparison at each run .
"""Runs the sampling multiple times and calculates a comparison at each run.

    Performs multiple runs of sampling and calculation of comparison metric.
    For each run, creates a "snapshot" sample of 1000 contributions to train a snapshot model on.
    Also creates a "test" sample of 1000 contributions for which to calculate a comparison, e.g. cross-entropy or KL-Divergence.
    Then calculates the cross entropy of each contribution in the test sample w.r.t the snapshot models.
    It does this calculation for every combination of groups.

    :param group_names: a list of the groups in the given data.
    :param all_group_contribs: a list of Dataframes of contributions - one for each group.
    :param all_groups_toks: a list that matches "all_group_contribs" but consists of a pandas Series of
                            tokenised contributions - one for each group.
    :param n_runs: the number of times to resample and recalculated cross entropy.
    :param comparison_func: the method used to compare the groups - must be "CE", "KLD", "CE_Fluct", or "KLD_Fluct".

    :returns: a list of dictionaries, one for each run. Each dictionary has another dictionary for each
                group which contains cross entropy results dataframes. One can find the information of a
                run as follows: "runs[group1][group2]" would contain a DataFrame with the cross entropies
                of test samples from group2 according to the snapshot model trained on group1.
    """ # Initialise empty output array. # Loop for each run. # Sample snapshot and test samples for each group. # Get all of the test tokens # Calculate the comparison values for all groups # Add dictionary for current run to output list.
a list of dictionaries, one for each run. Each dictionary has another dictionary for each
group which contains cross entropy results dataframes. One can find the information of a
run as follows: "runs[group1][group2]" would contain a DataFrame with the cross entropies
of test samples from group2 according to the snapshot model trained on group1.
a list of dictionaries one for each run . Each dictionary has another dictionary for each group which contains cross entropy results dataframes . One can find the information of a run as follows : " runs [ group1 ] [ group2 ] " would contain a DataFrame with the cross entropies of test samples from group2 according to the snapshot model trained on group1 .
group_names
a list of the groups in the given data.
a list of the groups in the given data .
all_groups_contribs
all_groups_toks
a list that matches "all_group_contribs" but consists of a pandas Series of
tokenised contributions - one for each group.
a list that matches " all_group_contribs " but consists of a pandas Series of tokenised contributions - one for each group .
n_runs
the number of times to resample and recalculated cross entropy.
the number of times to resample and recalculated cross entropy .
snap_samples_per_user
snap_num_users
test_sample_size
replace
n_words_per_contrib
win_func
win_size
win_step
comparison_method
all_group_contribs
a list of Dataframes of contributions - one for each group.
a list of Dataframes of contributions - one for each group .
comparison_func
the method used to compare the groups - must be "CE", "KLD", "CE_Fluct", or "KLD_Fluct".
the method used to compare the groups - must be " CE " " KLD " " CE_Fluct " or " KLD_Fluct " .