# To access the Travel Mode Choice data
import statsmodels.datasets

# To perform the dataset conversion
import pylogit as pl

"""
Explanation: Converting long-format dataframes to wide-format
The purpose of this notebook is to demonstrate the conversion of long-format data into wide-format. Long-format data contains one row per available alternative per choice situation. In contrast, wide-format data contains one row per choice situation. PyLogit and other software packages (e.g. mlogit in R) use data that is in long-format. However, other software packages, such as Statsmodels in Python or Python BIOGEME, use data that is in wide-format.
Because different software packages have different data format requirements, it is useful to be able to convert one's data from one format to another. Other PyLogit example notebooks (such as the "Main PyLogit Example") demonstrate how to take data from wide-format and convert it into long-format. This notebook will demonstrate the reverse process: taking data from long-format and converting it into wide-format.
The dataset being used in this example is the "Travel Mode Choice" dataset from Greene and Hensher. It is described on the statsmodels <a href="http://statsmodels.sourceforge.net/0.6.0/datasets/generated/modechoice.html">website</a>, and their description is reproduced below in full.
<pre>
    The data, collected as part of a 1987 intercity mode choice study, are a sub-sample of 210 non-business
    trips between Sydney, Canberra and Melbourne in which the traveler chooses a mode from four alternatives
    (plane, car, bus and train). The sample, 840 observations, is choice based with over-sampling of the
    less popular modes (plane, train and bus) and under-sampling of the more popular mode, car. The level of
    service data was derived from highway and transport networks in Sydney, Melbourne, non-metropolitan N.S.W.
    and Victoria, including the Australian Capital Territory.

    Number of observations: 840 Observations On 4 Modes for 210 Individuals.
    Number of variables: 8
    Variable name definitions::

        individual = 1 to 210
        mode =
            1 - air
            2 - train
            3 - bus
            4 - car
        choice =
            0 - no
            1 - yes
        ttme = terminal waiting time for plane, train and bus (minutes); 0
               for car.
        invc = in vehicle cost for all stages (dollars).
        invt = travel time (in-vehicle time) for all stages (minutes).
        gc = generalized cost measure:invc+(invt*value of travel time savings)
            (dollars).
        hinc = household income ($1000s).
        psize = traveling group size in mode chosen (number).


    Source

    Greene, W.H. and D. Hensher (1997) Multinomial logit and discrete choice models in Greene, W. H. (1997)
    LIMDEP version 7.0 user’s manual revised, Plainview, New York econometric software, Inc. Download from
    on-line complements to Greene, W.H. (2011) Econometric Analysis, Prentice Hall, 7th Edition (data table
    F18-2) http://people.stern.nyu.edu/wgreene/Text/Edition7/TableF18-2.csv

</pre>
End of explanation
"""


# Access the dataset
mode_data = statsmodels.datasets.modechoice.load_pandas()
# Get a pandas dataframe of the mode choice data
long_df = mode_data["data"]
# Look at the dataframe to ensure that it loaded correctly
long_df.head()

"""
Explanation: Load the needed dataset
End of explanation
"""


# ind_vars is a list of strings denoting the column
# headings of data that varies across choice situations,
# but not across alternatives. In our data, this is
# the household income and party size.
individual_specific_variables = ["hinc", "psize"]

# alt_specific_vaars is a list of strings denoting the
# column headings of data that vary not only across
# choice situations but also across all alternatives.
# These are columns such as the "level of service"
# variables.
alternative_specific_variables = ["invc", "invt", "gc"]

# subset_specific_vars is a dictionary. Each key is a
# string that denotes a variable that is subset specific.
# Each value is a list of alternative ids, over which the
# variable actually varies. Note that subset specific
# variables vary across choice situations and across some
# (but not all) alternatives. This is most common when
# using variables that are not meaningfully defined for
# all alternatives. An example of this in our dataset is
# terminal time ("ttme"). This variable is not meaningfully
# defined for the "car" alternative. Therefore, it is always
# zero. Note "4" is the id for the "car" alternative
subset_specific_variables = {"ttme": [1, 2, 3]}

# obs_id_col is the column denoting the id of the choice
# situation. If one was using a panel dataset, with multiple
# choice situations per unit of observation, the column
# denoting the unit of observation would be listed in
# ind_vars (i.e. with the individual specific variables)
observation_id_column = "individual"

# alt_id_col is the column denoting the id of the alternative
# corresponding to a given row.
alternative_id_column = "mode"

# choice_col is the column denoting whether the alternative
# on a given row was chosen in the corresponding choice situation
choice_column = "choice"

# Lastly, alt_name_dict is not necessary. However, it is useful.
# It records the names corresponding to each alternative, if there
# are any, and allows for the creation of meaningful column names
# in the wide-format data (such as when creating the columns
# denoting the available alternatives in each choice situation).
# The keys of alt_name_dict are the unique alternative ids, and
# the values are the names of each alternative.
alternative_name_dict = {1: "air",
                         2: "train",
                         3: "bus",
                         4: "car"}

"""
Explanation: Create the needed variables for the conversion function.
The function in PyLogit that is used to convert long-format data to wide-format data is "convert_long_to_wide," and it can be accessed through "pl.convert_long_to_wide". The docstring for the function contains all of the information necessary to perform the conversion, but we will leave it to readers to view the docstring at their own leisure. For now, we will simply create the needed objects/arguments for the function.
In particular, we will need the following 7 objects:
1. ind_vars
2. alt_specific_vars
3. subset_specific_vars
4. obs_id_col
5. alt_id_col
6. choice_col
7. alt_name_dict
The cells below will show exactly what these objects are.
End of explanation
"""


# Finally, we can create the wide format dataframe
wide_df = pl.convert_long_to_wide(long_df,
                                  individual_specific_variables,
                                  alternative_specific_variables,
                                  subset_specific_variables,
                                  observation_id_column,
                                  alternative_id_column,
                                  choice_column,
                                  alternative_name_dict)

# Let's look at the created dataframe, transposed for easy viewing
wide_df.head().T

"""
Explanation: Create the wide-format dataframe
End of explanation
"""


