!conda create -n asim python=3.9 activitysim -c conda-forge --override-channels

"""
Explanation: Getting Started with ActivitySim
This getting started guide is a Jupyter notebook. It is an interactive Python 3 environment that describes how to set up, run, and begin to analyze the results of ActivitySim modeling scenarios. It is assumed users of ActivitySim are familiar with the basic concepts of activity-based modeling.  This tutorial covers:

Installation and setup
Setting up and running a base model
Inputs and outputs
Setting up and running an alternative scenario
Comparing results
Next steps and further reading

This notebook depends on Anaconda Python 3 64bit.
Install ActivitySim
The first step is to install activitysim from conda forge.  This also installs dependent packages such as tables for reading/writing HDF5, openmatrix for reading/writing OMX matrix, and pyyaml for yaml settings files.  The command below also creates an asim conda environment just for activitysim.
End of explanation
"""


!conda activate asim

"""
Explanation: Activate the Environment
End of explanation
"""


!activitysim create -e example_mtc -d example
%cd example

"""
Explanation: Creating an Example Setup
The example is included in the package and can be copied to a user defined location using the package's command line interface.  The example includes all model steps.  The command below copies the example_mtc example to a new example folder.  It also changes into the new example folder so we can run the model from there.
End of explanation
"""


!activitysim run -c configs -d data -o output

"""
Explanation: Run the Example
The code below runs the example, which runs in a few minutes.  The example consists of 100 synthetic households and the first 25 zones in the example model region.  The full example (example_mtc_full) can be created and downloaded from the activitysim resources repository using activitysim's create command above.  As the model runs, it logs information to the screen.  
To run the example, use activitysim's built-in run command.  As shown in the script help, the default settings assume a configs, data, and output folder in the current directory.
End of explanation
"""


import os
for root, dirs, files in os.walk(".", topdown=False):
   for name in files:
      print(os.path.join(root, name))
   for name in dirs:
      print(os.path.join(root, name))

"""
Explanation: Inputs and Outputs Overview
An ActivitySim model requires:

Configs: settings, model step expressions files, etc.​
settings.yaml - main settings file for running the model
network_los.yaml - network level-of-service (skims) settings file
[model].yaml - configuration file for the model step (such as auto ownership)
[model].csv - expressions file for the model step
Data: input data - input data tables and skims​
land_use.csv - zone data file
households.csv - synthethic households
persons.csv - synthethic persons
skims.omx - all skims in one open matrix file
Output: output data - output data, tables, tracing info, etc.
pipeline.h5 - data pipeline database file (all tables at each model step)
final_[table].csv - final household, person, tour, trip CSV tables
activitysim.log - console log file
trace.[model].csv - trace calculations for select households
simulation.py: main script to run the model

Run the command below to list the example folder contents.
End of explanation
"""


print("Load libraries.")
import pandas as pd
import openmatrix as omx
import yaml
import glob

print("Display the settings file.\n")

with open(r'configs/settings.yaml') as file:
    file_contents = yaml.load(file, Loader=yaml.FullLoader)
    print(yaml.dump(file_contents))

print("Display the network_los file.\n")

with open(r'configs/network_los.yaml') as file:
    file_contents = yaml.load(file, Loader=yaml.FullLoader)
    print(yaml.dump(file_contents))

print("Input land_use.  Primary key: TAZ.  Required additional fields depend on the downstream submodels (and expression files).")
pd.read_csv("data/land_use.csv")

print("Input households.  Primary key: HHID.  Foreign key: TAZ.  Required additional fields depend on the downstream submodels (and expression files).")
pd.read_csv("data/households.csv")

print("Input persons.  Primary key: PERID.  Foreign key: household_id.  Required additional fields depend on the downstream submodels (and expression files).")
pd.read_csv("data/persons.csv")

print("Skims.  All skims are input via one OMX file.  Required skims depend on the downstream submodels (and expression files).\n")
print(omx.open_file("data/skims.omx"))

"""
Explanation: Inputs
Run the commands below to: 
* Load required Python libraries for reading data
* Display the settings.yaml, including the list of models to run
* Display the land_use, households, and persons tables
* Display the skims
End of explanation
"""


print("The output pipeline contains the state of each table after each model step.")
pipeline = pd.io.pytables.HDFStore('output/pipeline.h5')
pipeline.keys()

print("Households table after trip mode choice, which contains several calculated fields.")
pipeline['/households/joint_tour_frequency'] #watch out for key changes if not running all models

print("Final output households table to written to CSV, which is the same as the table in the pipeline.")
pd.read_csv("output/final_households.csv")

print("Final output persons table to written to CSV, which is the same as the table in the pipeline.")
pd.read_csv("output/final_persons.csv")

print("Final output tours table to written to CSV, which is the same as the table in the pipeline.  Joint tours are stored as one record.")
pd.read_csv("output/final_tours.csv")

print("Final output trips table to written to CSV, which is the same as the table in the pipeline.  Joint trips are stored as one record")
pd.read_csv("output/final_trips.csv")

"""
Explanation: Outputs
Run the commands below to: 
* Display the output household and person tables
* Display the output tour and trip tables
End of explanation
"""


print("Final output accessibility table to written to CSV.")
pd.read_csv("output/final_accessibility.csv")

print("Joint tour participants table, which contains the person ids of joint tour participants.")
pipeline['joint_tour_participants/joint_tour_participation']

print("Destination choice sample logsums table for school location if want_dest_choice_sample_tables=True.")
if '/school_location_sample/school_location' in pipeline:
    pipeline['/school_location_sample/school_location']

"""
Explanation: Other notable outputs
End of explanation
"""


print("trip matrices by time of day for assignment")
output_files = os.listdir("output")
for output_file in output_files:
    if "omx" in output_file:
        print(output_file)

"""
Explanation: Trip matrices
A write_trip_matrices step at the end of the model adds boolean indicator columns to the trip table in order to assign each trip into a trip matrix and then aggregates the trip counts and writes OD matrices to OMX (open matrix) files.  The coding of trips into trip matrices is done via annotation expressions.
End of explanation
"""


print("All trace files.\n")
glob.glob("output/trace/*.csv")


print("Trace files for auto ownership.\n")
glob.glob("output/trace/auto_ownership*.csv")

print("Trace chooser data for auto ownership.\n")
pd.read_csv("output\\trace\\auto_ownership_simulate.simple_simulate.eval_mnl.choosers.csv")

print("Trace utility expression values for auto ownership.\n")
pd.read_csv("output\\trace\\auto_ownership_simulate.simple_simulate.eval_mnl.eval_utils.expression_values.csv")

print("Trace alternative total utilities for auto ownership.\n")
pd.read_csv("output\\trace\\auto_ownership_simulate.simple_simulate.eval_mnl.utilities.csv")

print("Trace alternative probabilities for auto ownership.\n")
pd.read_csv("output\\trace\\auto_ownership_simulate.simple_simulate.eval_mnl.probs.csv")

print("Trace random number for auto ownership.\n")
pd.read_csv("output\\trace\\auto_ownership_simulate.simple_simulate.eval_mnl.rands.csv")

print("Trace choice for auto ownership.\n")
pd.read_csv("output\\trace\\auto_ownership_simulate.simple_simulate.eval_mnl.choices.csv")

"""
Explanation: Tracing calculations
Tracing calculations is an important part of model setup and debugging.  Often times data issues, such as missing values in input data and/or incorrect submodel expression files, do not reveal themselves until a downstream submodels fails.  There are two types of tracing in ActivtiySim: household and origin-destination (OD) pair. If a household trace ID is specified via trace_hh_id, then ActivitySim will output a comprehensive set of trace files for all calculations for all household members.  These trace files are listed below and explained.
End of explanation
"""


!activitysim run -c configs_mp -c configs -d data -o output

"""
Explanation: Run the Multiprocessor Example
The command below runs the multiprocessor example, which runs in a few minutes.  It uses settings inheritance to override setings in the configs folder with settings in the configs_mp folder.  This allows for re-using expression files and settings files in the single and multiprocessed setups.  The multiprocessed example uses the following additional settings:
```
num_processes: 2
chunk_size: 0
chunk_training_mode: disabled
multiprocess_steps:
  - name: mp_initialize
    begin: initialize_landuse
  - name: mp_households
    begin: school_location
    slice:
      tables:
        - households
        - persons
  - name: mp_summarize
    begin: write_data_dictionary
```
In brief, num_processes specifies the number of processors to use and a chunk_size of 0 plus a chunk_training_mode of disabled means ActivitySim is free to use all the available RAM if needed.  The multiprocess_steps specifies the beginning, middle, and end steps in multiprocessing.  The mp_initialize step is single processed because there is no slice setting.  It starts with the initialize_landuse submodel and runs until the submodel identified by the next multiprocess submodel starting point, school_location.  The mp_households step is multiprocessed and the households and persons tables are sliced and allocated to processes using the chunking settings.  The rest of the submodels are run multiprocessed until the final multiprocess step.  The mp_summarize step is single processed because there is no slice setting and it writes outputs.  See multiprocessing and chunk_size for more information.
End of explanation
"""


