import pandas as pd
import numpy as np

"""
Explanation: Recap
In order of priority/time taken

basalareaincremementnonspatialaw
this is actually slow because of the number of times the BAFromZeroToDataAw function is called as shown above
relaxing the tolerance may help
indeed the tolerance is 0.01 * some value while the other factor finder functions have 0.1 tolerance i think
can also use cython for the increment functions


vectorize merch and gross volume functions
they require a lot of getting scalars off data frame, which is quite slow. faster to get an array



do a profiling run with IO (of reading input data and writing the plot curves to files) in next run
Decide on the action

speed up increment functions
use cython for increment functions
it turns out this may not help that much. the function is pretty fast, it's called almost 500,000 times on the sample of 300 plots
reduce the number of times its called maybe by using gradient descent for the optimization?
relax the tolerance
gives a context to refactor them as well (into their own module) which would be a welcome change
the increment functions use numpy functions but operate on scalars, there is no benefit to using numpy functions there


performance-wise, it is not clear that this will pay off so much. vectorizing the volume functions is probably wiser

Characterize what is happening
End of explanation
"""


from gypsy.GYPSYNonSpatial import GrossTotalVolume_Pl


GrossTotalVolume_Pl(np.random.random(10) * 100, np.random.random(10) * 100)

"""
Explanation: The original gross volume function checks that top height is greater than 0
``` python
def GrossTotalVolume_Pl(BA_Pl, topHeight_Pl):
    Tvol_Pl = 0
if topHeight_Pl &gt; 0:
    a1 = 0.194086
    a2 = 0.988276
    a3 = 0.949346
    a4 = -3.39036
    Tvol_Pl = a1* (BA_Pl**a2) * (topHeight_Pl **a3) * numpy.exp(1+(a4/((topHeight_Pl**2)+1)))

return Tvol_Pl

```
This makes it fail if trying to use it on an array:
End of explanation
"""


def GrossTotalVolume_Pl_arr(BA_Pl, topHeight_Pl):
    a1 = 0.194086
    a2 = 0.988276
    a3 = 0.949346
    a4 = -3.39036
    Tvol_Pl = a1* (BA_Pl**a2) * (topHeight_Pl **a3) * np.exp(1+(a4/((topHeight_Pl**2)+1)))

    return Tvol_Pl

print(GrossTotalVolume_Pl_arr(10, 10))
print(GrossTotalVolume_Pl_arr(0, 10))
print(GrossTotalVolume_Pl_arr(10, 0))
print(GrossTotalVolume_Pl_arr(np.random.random(10) * 100, np.random.random(10) * 100))
print(GrossTotalVolume_Pl_arr(np.zeros(10) * 100, np.random.random(10) * 100))

"""
Explanation: MWEs
If we can rewrite it to handle 0s properly, i.e. to return 0 where an input is 0, then it is trivial to vectorize
End of explanation
"""


ba = np.random.random(1000) * 100
top_height = np.random.random(1000) * 100
d = pd.DataFrame({'ba': ba, 'th': top_height})

%%timeit
d.apply(
    lambda x: GrossTotalVolume_Pl(
        x.at['ba'],
        x.at['th']
    ),
    axis=1
)

%%timeit
GrossTotalVolume_Pl_arr(ba, top_height)

"""
Explanation: Timings
End of explanation
"""


%%bash
git log --since "2016-11-14 19:30" --oneline # 19:30 GMT/UTC

! git diff "HEAD~$(git log --since "2016-11-14 19:30" --oneline | wc -l)" ../gypsy

"""
Explanation: The array method is 20x faster. This is worth implementing. We should also add tests to help be explicity about the behaviour of these volume functions.
Revise the code
Go on. Do it.
Tests
Yes, though data was changed as the new implementations yield NaN where input is NaN, instead of yielding 0.
Review code changes
End of explanation
"""


%%bash
# git checkout 36941343aca2df763f93192abef461093918fff4 -b vectorize-volume-functions
# time gypsy simulate ../private-data/prepped_random_sample_300.csv --output-dir tmp
# rm -rfd tmp

# real	4m51.287s
# user	4m41.770s
# sys	0m1.070s

45/336.

"""
Explanation: Run timings
From last time:
real    5m36.407s
user    5m25.740s
sys     0m2.140s
After cython'ing iter functions:
End of explanation
"""


from gypsy.forward_simulation import simulate_forwards_df

data = pd.read_csv('../private-data/prepped_random_sample_300.csv', index_col=0, nrows=10)

%%prun -D forward-sim-3.prof -T forward-sim-3.txt -q
result = simulate_forwards_df(data)

!head forward-sim-3.txt

"""
Explanation: It yielded a 13% reduction in the time.
Run profiling
End of explanation
"""


! rm -rfd gypsy-output

output_dir = 'gypsy-output'

%%prun -D forward-sim-2.prof -T forward-sim-2.txt -q
# restart the kernel first
data = pd.read_csv('../private-data/prepped_random_sample_300.csv', index_col=0, nrows=10)
result = simulate_forwards_df(data)
os.makedirs(output_dir)
for plot_id, df in result.items():
    filename = '%s.csv' % plot_id
    output_path = os.path.join(output_dir, filename)
    df.to_csv(output_path)


"""
Explanation: Compare performance visualizations
Now use either of these commands to visualize the profiling
```
pyprof2calltree -k -i forward-sim-1.prof forward-sim-3.txt
or
dc run --service-ports snakeviz notebooks/forward-sim-3.prof
```
Old

New

Summary of performance improvements
The calculation of gross and merchantable volume is drastically faster now; under profiling it decrease to 1 second from 22 seconds.
A lot of that seems to be profiler overhead, as when using gypsy simulate CLI it only got 15% faster; however I expect i/o is obfuscating the outcome there.
Profile with I/O
End of explanation
"""


