%%bash

cat /data/GLM-wkshp/flashsort/environment_setup.README

# Links to representative PDFs.
from IPython.display import display, HTML, Image
class PDF(object):
    def __init__(self, filename):
        self.filename = filename

    def _repr_pdf_(self):
        return open(self.filename, 'rb').read()

base_path = '/data/GLM-wkshp/flashsort/figures-length/IOPsupercell18-AL-20090410-boundingbox-thresh-0.15_dist-3000.0_pts-10/'


"""
Explanation: See environment_setup.README (below) for instructions about the use of the DC3_plots_NALMA script. It is a version of the script used to process the DC3 dataset as in Barth et al. (2015, BAMS) and Bruning and Thomas (2015, JGR).
The flash sorting infrastructure is modular. This script uses the <a href="http://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html#sklearn.cluster.DBSCAN">DBSCAN algorithm </a> as implemented in the <a href="http://scikit-learn.org">scikit-learn</a> machine-learning library. In order to manage the $N^2$ efficiency of the underlying DBSCAN implementation, data are clustered in pairs of thresh_duration chunks.
The script is configurable in a few places. 
- base_sort_dir sets the path where 
- center_ID chooses a network center. The centers are defined in the centers dictionary. The ID is used later when constructing output filenames, too.
- The params dictionary configures the flash sorting algorithm. Of particular importance are the following.
  - stations: sets the (min, max) number of stations that must participate in each solution for it to count. Max should be larger than the number of stations. Min should be six or seven, depending on the number of stations.
  - chi2: sets the (min, max) chi-squared value. The minimum should be zero, while a good maximum to start with is 1.0.
  - distance: maximum distance between a source and its closest neighbor before a new flash is started
  - thresh_critical_time: maximum temporal separation between a source and its closest neighbor before a new flash is started
  - thresh_duration: All flashes should be last less than or equal to this number of seconds. All flashes of duration < thresh_duration are guaranteed to remain clustered. An occasional lucky flash of duration =  2 * thresh_duration is possible.
The script is broken into three sections.
- Run the flash sorting, which creates HDF5 data files with VHF source data, their flash IDs, and a matching flash data table.
- Grab the flash-sorted files and create CF-compliant NetCDF grids
- Grab the grids and create PDF images of each grid
The grid spacing, boundaries, and frame intervals are configured at the begining of the gridding section of the script. This script creates regularly-spaced lat/lon grids, with the center grid cell size calculated to match the specified dx_km and dy_km. It is also possible to grid directly in a map projection of choice by changing proj_name, as well as x_name and y_name in the call to make_plot. For instance, a geostationary projection can be obtained with proj='geos' as described in the documentation for the proj4 coordinate system library.
The PDF images are created as small-multiple plots, with the number of columns given by n_cols at the beginning of the plotting section.
An example of reading and working with the resulting data files is found in the "Reading the flash-sorted files.ipynb"
As described below, additional scripts perform follow-on analysis.
- Assigning NLDN strokes to the best-matching flash
- Using a storm cell or storm region polygon to subset some flashes from the data files.
    - Creating time series plots of moments of the flash size distribution
    - Creating ASCII files of flash size and rate statistics
The IOP bounding box file included here is a rectangular lat/lon box, but the underlying code works with arbitrary polygons. Adapting the existing code to polygons is mostly a matter of reading in polygon vertices and sending its vertices instead of those for a rectangle.
End of explanation
"""


Image(base_path+'flashes.png')

"""
Explanation: Case-specific time series data
 Locations of all flashes contributing to the analyses 
Lets you identify that only one cell was tracked
End of explanation
"""


PDF(base_path+'D-1.7_b-0.25_length-profiles_CG.pdf')

PDF(base_path+'D-1.7_b-0.25_length-profiles_IC.pdf')

"""
Explanation: Time-height plot, separated by IC and CG.
There are accompanying ASCII data files with the raw data. Uses the technique in Bruning and Thomas (2015, JGR)
End of explanation
"""


PDF(base_path+'moment-energy-timeseries.pdf')

import pandas as pd
pd.read_csv(base_path+'../IOPsupercell18-output.flash_stats.csv')


"""
Explanation: Time series of flash moments 
There are accompanying ASCII data files with the raw data. Uses the technique in Bruning and Thomas (2015, JGR).
End of explanation
"""


Image(base_path+'/grids_lma_source/lma_source_20090410_185300.png')

Image(base_path+'/grids_flash_extent/flash_extent_20090410_185300.png')

Image(base_path+'/grids_flash_initiation/flash_initiation_20090410_185300.png')

Image(base_path+'/grids_flash_footprint/flash_footprint_20090410_185300.png')

"""
Explanation: Plots of each minute in the gridded data 
LMA source density, flash extent density, flash initation density, and average flash area.
End of explanation
"""


pd.read_csv(base_path+'/grids_flash_extent/flash_extent_20090410.csv')

"""
Explanation: Each of the grid type folders contains a CSV file with statistics of the pixels making up the image.
End of explanation
"""


PDF(base_path+'LYLOUT_090410_180000_3600-energy.pdf')

"""
Explanation: Flash energy spectra 
Plots of the flash energy spectra as defined in Bruning and MacGorman (2013, JAS). A 5/3 power law reference line is plotted.
End of explanation
"""


