# Configure Jupyter so figures appear in the notebook
%matplotlib inline

# Configure Jupyter to display the assigned value after an assignment
%config InteractiveShell.ast_node_interactivity='last_expr_or_assign'

# import functions from the modsim.py module
from modsim import *

"""
Explanation: Modeling and Simulation in Python
Chapter 5
Copyright 2017 Allen Downey
License: Creative Commons Attribution 4.0 International
End of explanation
"""


from pandas import read_html

"""
Explanation: Reading data
Pandas is a library that provides tools for reading and processing data.  read_html reads a web page from a file or the Internet and creates one DataFrame for each table on the page.
End of explanation
"""


filename = 'data/World_population_estimates.html'
tables = read_html(filename, header=0, index_col=0, decimal='M')
len(tables)

"""
Explanation: The data directory contains a downloaded copy of https://en.wikipedia.org/wiki/World_population_estimates
The arguments of read_html specify the file to read and how to interpret the tables in the file.  The result, tables, is a sequence of DataFrame objects; len(tables) reports the length of the sequence.
End of explanation
"""


table2 = tables[2]
table2.head()

"""
Explanation: We can select the DataFrame we want using the bracket operator.  The tables are numbered from 0, so tables[2] is actually the third table on the page.
head selects the header and the first five rows.
End of explanation
"""


table2.tail()

"""
Explanation: tail selects the last five rows.
End of explanation
"""


table2.columns = ['census', 'prb', 'un', 'maddison', 
                  'hyde', 'tanton', 'biraben', 'mj', 
                  'thomlinson', 'durand', 'clark']

"""
Explanation: Long column names are awkard to work with, but we can replace them with abbreviated names.
End of explanation
"""


table2.head()

"""
Explanation: Here's what the DataFrame looks like now.
End of explanation
"""


census = table2.census
census.head()

census.tail()

"""
Explanation: The first column, which is labeled Year, is special.  It is the index for this DataFrame, which means it contains the labels for the rows.
Some of the values use scientific notation; for example, 2.544000e+09 is shorthand for $2.544 \cdot 10^9$ or 2.544 billion.
NaN is a special value that indicates missing data.
Series
We can use dot notation to select a column from a DataFrame.  The result is a Series, which is like a DataFrame with a single column.
End of explanation
"""


un = table2.un / 1e9
un.head()

census = table2.census / 1e9
census.head()

"""
Explanation: Like a DataFrame, a Series contains an index, which labels the rows.
1e9 is scientific notation for $1 \cdot 10^9$ or 1 billion.
From here on, we will work in units of billions.
End of explanation
"""


plot(census, ':', label='US Census')
plot(un, '--', label='UN DESA')
    
decorate(xlabel='Year',
         ylabel='World population (billion)')

savefig('figs/chap05-fig01.pdf')

"""
Explanation: Here's what these estimates look like.
End of explanation
"""


max(abs(census - un) / un) * 100

"""
Explanation: The following expression computes the elementwise differences between the two series, then divides through by the UN value to produce relative errors, then finds the largest element.
So the largest relative error between the estimates is about 1.3%.
End of explanation
"""


# Solution goes here

# Solution goes here

# Solution goes here

# Solution goes here

"""
Explanation: Exercise: Break down that expression into smaller steps and display the intermediate results, to make sure you understand how it works.

Compute the elementwise differences, census - un
Compute the absolute differences, abs(census - un)
Compute the relative differences, abs(census - un) / un
Compute the percent differences, abs(census - un) / un * 100
End of explanation
"""


census[1950]

"""
Explanation: max and abs are built-in functions provided by Python, but NumPy also provides version that are a little more general.  When you import modsim, you get the NumPy versions of these functions.
Constant growth
We can select a value from a Series using bracket notation.  Here's the first element:
End of explanation
"""


census[2016]

"""
Explanation: And the last value.
End of explanation
"""


t_0 = get_first_label(census)

t_end = get_last_label(census)

elapsed_time = t_end - t_0

"""
Explanation: But rather than "hard code" those dates, we can get the first and last labels from the Series:
End of explanation
"""


p_0 = get_first_value(census)

p_end = get_last_value(census)

"""
Explanation: And we can get the first and last values:
End of explanation
"""


total_growth = p_end - p_0

annual_growth = total_growth / elapsed_time

"""
Explanation: Then we can compute the average annual growth in billions of people per year.
End of explanation
"""


results = TimeSeries()

"""
Explanation: TimeSeries
Now let's create a TimeSeries to contain values generated by a linear growth model.
End of explanation
"""


results[t_0] = census[t_0]
results

"""
Explanation: Initially the TimeSeries is empty, but we can initialize it so the starting value, in 1950, is the 1950 population estimated by the US Census.
End of explanation
"""


for t in linrange(t_0, t_end):
    results[t+1] = results[t] + annual_growth

"""
Explanation: After that, the population in the model grows by a constant amount each year.
End of explanation
"""


plot(census, ':', label='US Census')
plot(un, '--', label='UN DESA')
plot(results, color='gray', label='model')

decorate(xlabel='Year', 
         ylabel='World population (billion)',
         title='Constant growth')

savefig('figs/chap05-fig02.pdf')

"""
Explanation: Here's what the results looks like, compared to the actual data.
End of explanation
"""


# Solution goes here

census.loc[1960:1970]

"""
Explanation: The model fits the data pretty well after 1990, but not so well before.
Exercises
Optional Exercise:  Try fitting the model using data from 1970 to the present, and see if that does a better job.
Hint: 


Copy the code from above and make a few changes.  Test your code after each small change.


Make sure your TimeSeries starts in 1950, even though the estimated annual growth is based on later data.


You might want to add a constant to the starting value to match the data better.
End of explanation
"""


