%matplotlib inline

from matplotlib import rcParams
import matplotlib.pyplot as plt
import pandas as pd
import nilmtk
from nilmtk import DataSet, MeterGroup

plt.style.use('ggplot')
rcParams['figure.figsize'] = (13, 10)

redd = DataSet('/data/redd.h5')
elec = redd.buildings[1].elec
elec

"""
Explanation: MeterGroup, ElecMeter, selection and basic statistics
All NILM datasets consists of various groupings of electricity meters.  We can group the meters by house.  Or by the type of appliance they are directly connected to.  Or by sample rate.  Or by whether the meter is a whole-house "site meter" or an appliance-level submeter, or a circuit-level submeter.
In NILMTK, one of the key classes is MeterGroup which stores a list of meters and allows us to select a subset of meters, aggregate power from all meters and many other functions.
When we first open a DataSet, NILMTK creates several MeterGroup objects.  There's nilmtk.global_meter_group which holds every meter currently loaded (including from multiple datasets if you have opened more than one dataset).  There is also one MeterGroup per building (which live in the Building.elec attribute).  We also nested MeterGroups for aggregating together split-phase mains, 3-phase mains and dual-supply (240 volt) appliances in North American and Canadian datasets.  For example, here is the MeterGroup for building 1 in REDD:
NOTE: If you are on Windows, remember to escape the back-slashes, use forward-slashs, or use raw-strings when passing paths in Python, e.g. one of the following would work:
python
redd = DataSet('c:\\data\\redd.h5')
redd = DataSet('c:/data/redd.h5')
redd = DataSet(r'c:\data\redd.h5')
End of explanation
"""


elec.nested_metergroups()

"""
Explanation: Note that there are two nested MeterGroups: one for the electric oven, and one for the washer dryer (both of which are 240 volt appliances and have two meters per appliance):
End of explanation
"""


elec.mains()

"""
Explanation: Putting these meters into a MeterGroup allows us to easily sum together the power demand recorded by both meters to get the total power demand for the entire appliance (but it's also very easy to see the individual meter power demand too).
We can easily get a MeterGroup of either the submeters or the mains:
End of explanation
"""


elec.mains().power_series_all_data().head()

elec.submeters()

"""
Explanation: We can easily get the power data for both mains meters summed together:
End of explanation
"""


elec.proportion_of_energy_submetered()

"""
Explanation: Stats for MeterGroups
Proportion of energy submetered
Let's work out the proportion of energy submetered in REDD building 1:
End of explanation
"""


mains = elec.mains()

mains.available_ac_types('power')

elec.submeters().available_ac_types('power')

next(elec.load())

"""
Explanation: Note that NILMTK has raised a warning that Mains uses a different type of power measurement than all the submeters, so it's not an entirely accurate comparison.  Which raises the question: which type of power measurements are used for the mains and submeters?  Let's find out...
Active, apparent and reactive power
End of explanation
"""


elec.mains().total_energy() # returns kWh

"""
Explanation: Total Energy
End of explanation
"""


energy_per_meter = elec.submeters().energy_per_meter() # kWh, again
energy_per_meter

"""
Explanation: Energy per submeter
End of explanation
"""


# energy_per_meter is a DataFrame where each row is a 
# power type ('active', 'reactive' or 'apparent').
# All appliance meters in REDD are record 'active' so just select
# the 'active' row:
energy_per_meter = energy_per_meter.loc['active']
more_than_20 = energy_per_meter[energy_per_meter > 20]
more_than_20

instances = more_than_20.index
instances

"""
Explanation: column headings are the ElecMeter instance numbers.
The function fraction_per_meter does the same thing as energy_per_submeter but returns the fraction of energy per meter.
Select meters on the basis of their energy consumption
Let's make a new MeterGroup which only contains the ElecMeters which used more than 20 kWh:
End of explanation
"""


fraction = elec.submeters().fraction_per_meter().dropna()

# Create convenient labels
labels = elec.get_labels(fraction.index)
plt.figure(figsize=(10,30))
fraction.plot(kind='pie', labels=labels);

"""
Explanation: Plot fraction of energy consumption of each appliance
End of explanation
"""


elec.draw_wiring_graph()

"""
Explanation: Draw wiring diagram
We can get the wiring diagram for the MeterGroup:
End of explanation
"""


elec.meters_directly_downstream_of_mains()

"""
Explanation: It's not very pretty but it shows that meters (1,2) (the site meters) are upstream of all other meters.
Buildings in REDD have only two levels in their meter hierarchy (mains and submeters).  If there were more than two levels then it might be useful to get only the meters immediately downstream of mains:
End of explanation
"""


#sns.set_palette("Set3", n_colors=12)
# Set a threshold to remove residual power noise when devices are off
elec.plot_when_on(on_power_threshold = 40)

"""
Explanation: Plot appliances when they are in use
End of explanation
"""


fridge_meter = elec['fridge']

"""
Explanation: Stats and info for individual meters
The ElecMeter class represents a single electricity meter.  Each ElecMeter has a list of associated Appliance objects.  ElecMeter has many of the same stats methods as MeterGroup such as total_energy and available_power_ac_types and power_series and power_series_all_data.  We will now explore some more stats functions (many of which are also available on MeterGroup)...
End of explanation
"""


fridge_meter.upstream_meter() # happens to be the mains meter group!

"""
Explanation: Get upstream meter
End of explanation
"""


fridge_meter.device

"""
Explanation: Metadata about the class of meter
End of explanation
"""


fridge_meter.dominant_appliance()

"""
Explanation: Dominant appliance
If the metadata specifies that a meter has multiple meters connected to it then one of those can be specified as the 'dominant' appliance, and this appliance can be retrieved with this method:
End of explanation
"""


fridge_meter.total_energy() # kWh

"""
Explanation: Total energy
End of explanation
"""


fridge_meter.plot()

"""
Explanation: Get good sections
If we plot the raw power data then we see there is one large gap where, supposedly, the metering system was not working.  (if we were to zoom in then we'd see lots of smaller gaps too):
End of explanation
"""


good_sections = fridge_meter.good_sections(full_results=True)
# specifying full_results=False would give us a simple list of 
# TimeFrames.  But we want the full GoodSectionsResults object so we can
# plot the good sections...

good_sections.plot()

"""
Explanation: We can automatically identify the 'good sections' (i.e. the sections where every pair of consecutive samples is less than max_sample_period specified in the dataset metadata):
End of explanation
"""


good_sections.combined()

"""
Explanation: The blue chunks show where the data is good.  The white gap is the large gap seen in the raw power data.  There are lots of smaller gaps that we cannot see at this zoom level.
We can also see the exact sections identified:
End of explanation
"""


fridge_meter.dropout_rate()

"""
Explanation: Dropout rate
As well as large gaps appearing because the entire system is down, we also get frequent small gaps from wireless sensors dropping data.  This is sometimes called 'dropout'.  The dropout rate is a number between 0 and 1 which specifies the proportion of missing samples.  A dropout rate of 0 means no samples are missing.  A value of 1 would mean all samples are missing:
End of explanation
"""


import nilmtk
nilmtk.global_meter_group.select_using_appliances(type='washer dryer')

"""
Explanation: Note that the dropout rate has gone down (which is good!) now that we are ignoring the gaps.  This value is probably more representative of the performance of the wireless system.
Select subgroups of meters
We use ElecMeter.select_using_appliances() to select a new MeterGroup using an metadata field.  For example, to get all the washer dryers in the whole of the REDD dataset:
End of explanation
"""


elec.select_using_appliances(type=['fridge', 'microwave'])

"""
Explanation: Or select multiple appliance types:
End of explanation
"""


nilmtk.global_meter_group.select_using_appliances(category='heating')

"""
Explanation: Or all appliances in the 'heating' category:
End of explanation
"""


nilmtk.global_meter_group.select_using_appliances(building=1, category='single-phase induction motor')

"""
Explanation: Or all appliances in building 1 with a single-phase induction motor(!):
End of explanation
"""


nilmtk.global_meter_group.select_using_appliances(building=2, category='laundry appliances')

"""
Explanation: (NILMTK imports the 'common metadata' from the NILM Metadata project, which includes a wide range of different category taxonomies)
End of explanation
"""


elec.select(device_model='REDD_whole_house')

elec.select(sample_period=3)

"""
Explanation: Select a group of meters from properties of the meters (not the appliances)
End of explanation
"""


elec['fridge']

"""
Explanation: Select a single meter from a MeterGroup
We use [] to retrive a single ElecMeter from a MeterGroup.
Search for a meter using appliances connected to each meter
End of explanation
"""


elec.select_using_appliances(type='fridge')

elec['light', 2]

"""
Explanation: Appliances are uniquely identified within a building by a type (fridge, kettle, television, etc.) and an instance number.  If we do not specify an instance number then ElecMeter retrieves instance 1 (instance numbering starts from 1).  If you want a different instance then just do this:
End of explanation
"""


import nilmtk
nilmtk.global_meter_group[{'dataset': 'REDD', 'building': 1, 'type': 'fridge', 'instance': 1}]

"""
Explanation: To uniquely identify an appliance in nilmtk.global_meter_group then we must specify the dataset name, building instance number, appliance type and appliance instance in a dict:
End of explanation
"""


elec[1]

"""
Explanation: Search for a meter using details of the ElecMeter
get ElecMeter with instance = 1:
End of explanation
"""


from nilmtk.elecmeter import ElecMeterID 
# ElecMeterID is a namedtuple for uniquely identifying each ElecMeter

nilmtk.global_meter_group[ElecMeterID(instance=8, building=1, dataset='REDD')]

"""
Explanation: Instance numbering
ElecMeter and Appliance instance numbers uniquely identify the meter or appliance type within the building, not globally.  To uniquely identify a meter globally, we need three keys:
End of explanation
"""


elec[[ElecMeterID(instance=3, building=1, dataset='REDD'), 
      ElecMeterID(instance=4, building=1, dataset='REDD')]]

elec[ElecMeterID(instance=(3,4), building=1, dataset='REDD')]

"""
Explanation: Select nested MeterGroup
We can also select a single, existing nested MeterGroup.  There are two ways to specify a nested MeterGroup:
End of explanation
"""


elec[ElecMeterID(instance=0, building=1, dataset='REDD')]

"""
Explanation: We can also specify the mains by asking for meter instance 0:
End of explanation
"""


elec.mains() == elec[ElecMeterID(instance=0, building=1, dataset='REDD')]

"""
Explanation: which is equivalent to elec.mains():
End of explanation
"""


redd.set_window(start='2011-04-21', end='2011-04-22')
elec.plot();
plt.xlabel("Time");

"""
Explanation: Plot sub-metered data for a single day
End of explanation
"""


from pandas.plotting import autocorrelation_plot

elec.mains().plot_autocorrelation();

"""
Explanation: Autocorrelation Plot
End of explanation
"""


fridges_restricted = nilmtk.global_meter_group.select_using_appliances(type='fridge')
daily_energy = pd.Series([meter.average_energy_per_period(offset_alias='D') 
                          for meter in fridges_restricted.meters])

# daily_energy.plot(kind='hist');
# plt.title('Histogram of daily fridge energy');
# plt.xlabel('energy (kWh)');
# plt.ylabel('occurences');
# plt.legend().set_visible(False)

daily_energy

"""
Explanation: Daily energy consumption across fridges in the dataset
End of explanation
"""


correlation_df = elec.pairwise_correlation()
correlation_df

"""
Explanation: Correlation dataframe of the appliances
End of explanation
"""


