f = lambda x: 2 * x
f(5)

"""
Explanation: Event Data Filtering
by: Sebastiaan J. van Zelst
Like any data-driven field, the successful application of process mining needs data munging and crunching.
In pm4py, you can munge and crunch your data in two ways, i.e., you can write lambda functions and apply them on
your event log, or, you can apply pre-built filtering and transformation functions.
Hence, in this turtorial, we briefly explain how to filter event data in various different ways in pm4py.
Generic Lambda Functions
In a nutshell, a lambda function allows you to specify a function that needs to be applied on a given element.
As a simple example, consider the following snippet:
End of explanation
"""


f('Pete')

"""
Explanation: In the code, we assign a lambda function to variable f.
The function specifies that on each possible input it receives, the resulting function that is applied is a multiplication by 2.
Hence f(1)=2, f(2)=4, etc.
Note that, invoking f only works if we provide an argument that can be combined with the * 2 operation.
For example, for strings, the * 2 operation concatenates the input argument with itself:
End of explanation
"""


l = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
filter(lambda n: n >= 5, l)

"""
Explanation: Filter and Map
Lambda functions allow us to write short, type-independent functions.
Given a list of objects, Python provides two core functions that can apply a given lambda function on each element of
the given list (in fact, any iterable):

filter(f,l)
apply the given lambda function f as a filter on the iterable l.
map(f,l)
apply the given lambda function f as a transformation on the iterable l.

For more information, study the concept of ‘higher order functions’ in Python, e.g., as introduced here.
Let's consider a few simple examples.
End of explanation
"""


list(filter(lambda n: n >= 5, l))

"""
Explanation: The previous example needs little to no explanation, i.e., the filter retains all numbers in the list greater or equal to five.
However, what is interesting, is the fact that the resulting objects are not a list (or an iterables), rather a filter object.
Such an objects can be easily transformed to a list by wrapping it with a list() cast:
End of explanation
"""


map(lambda n: n * 3, l)

list(map(lambda n: n * 3, l))

"""
Explanation: The same holds for the map() function:
End of explanation
"""


import pm4py

log = pm4py.read_xes('data/running_example.xes')
# inspect the length of each trace using a generic map function
list(map(lambda t: len(t), log))

"""
Explanation: Observe that, the previous map function simply muliplies each element of list l by three.
Lambda-Based Filtering in pm4py
In pm4py, event log objects mimic lists of traces, which in turn, mimic lists of events.
Clearly, lambda functions can therefore be applied to event logs and traces.
However, as we have shown in the previous example, after applying such a lamda-based filter, the resulting object is no longer an event log.
Furthermore, casting a filter object or map object to an event log in pm4py is a bit more involved, i.e., it is
not so trivial as list(filter(...)) in the previous example.
This is due to the fact that various meta-data is stored in the event log object as well.
To this end, pm4py offers wrapper functions that make sure that after applying your higher-order function with a lambda function,
the resulting object is again an Event Log object.
In the upcoming scripts, we'll take a look at some lambda-based fitlering.
First, let's inspect the length of each trace in our running example log by applying a generic map function
End of explanation
"""


lf = pm4py.filter_log(lambda t: len(t) > 5, log)
list(map(lambda t: len(t), lf))

"""
Explanation: As we can see, there are four traces describing a trace of length 5, one trace of length 9 and one trace of length 13.
Let's retain all traces that have a lenght greater than 5.
End of explanation
"""


list(map(lambda t: (len(t), len(list(filter(lambda e: e['concept:name'] == 'reinitiate request', t)))), log))

"""
Explanation: The traces of length 9 and 13 have repeated behavior in them, i.e., the reinitiate request activity has been performed at least once:
End of explanation
"""


print(len(log))
lf = pm4py.filter_log(lambda t: len(t) > 5, log)
print(len(lf))

print(len(log[0]))  #log[0] fetches the 1st trace
tf = pm4py.filter_trace(lambda e: e['concept:name'] in {'register request', 'pay compensation'}, log[0])
print(len(tf))

print(len(log[0]))
ls = pm4py.sort_log(log, lambda t: len(t))
print(len(ls[0]))
ls = pm4py.sort_log(log, lambda t: len(t), reverse=True)
print(len(ls[0]))

"""
Explanation: Observe that the map function maps each trace onto a tuple.
The first element describes the length of the trace.
The second element describes the number of occurrences of the activity register request.
Observe that we obtain said counter by filtering the trace, i.e., by retaining only those events that describe the
reinitiate request activity and counting the length of the resulting list.
Note that the traces describe a list of events, and, events are implementing a dictionary.
In this case, the activity name is captured by the concept:name attribute.
In general, PM4PY supports the following generic filtering functions:

pm4py.filter_log(f, log)
filter the log according to a function f.
pm4py.filter_trace(f,trace)
filter the trace according to function f.
pm4py.sort_log(log, key, reverse)
sort the event log according to a given key, reversed order if reverse==True.
pm4py.sort_trace(trace, key, reverse)
sort the trace according to a given key, reversed order if reverse==True.

Let's see these functions in action:
End of explanation
"""


pm4py.filter_start_activities(log, {'register request'})

pm4py.filter_start_activities(log, {'register request TYPO!'})

import pandas

ldf = pm4py.format_dataframe(pandas.read_csv('data/running_example.csv', sep=';'), case_id='case_id',
                             activity_key='activity', timestamp_key='timestamp')
pm4py.filter_start_activities(ldf, {'register request'})

pm4py.filter_start_activities(ldf, {'register request TYPO!'})

"""
Explanation: Specific Filters
There are various pre-built filters in PM4Py, which make commonly needed process mining filtering functionality a lot easier.
In the upcoming overview, we briefly give present these functions.
We describe how to call them, their main input parameters and their return objects.
Note that, all of the filters work on both DataFrames and pm4py event log objects.
Start Activities

filter_start_activities(log, activities, retain=True)
retains (or drops) the traces that contain the given activity as the final event.
End of explanation
"""


len(pm4py.filter_end_activities(log, 'pay compensation'))

"""
Explanation: End Activities

filter_end_activities(log, activities, retain=True)
retains (or drops) the traces that contain the given activity as the final event.

For example, we can retain the number of cases that end with a "payment of the compensation":
End of explanation
"""


# retain any case that has either Peter or Mike working on it
lf = pm4py.filter_event_attribute_values(log, 'org:resource', {'Pete', 'Mike'})
list(map(lambda t: list(map(lambda e: e['org:resource'], t)), lf))

# retain only those events that have Pete or Mik working on it
lf = pm4py.filter_event_attribute_values(log, 'org:resource', {'Pete', 'Mike'}, level='event')
list(map(lambda t: list(map(lambda e: e['org:resource'], t)), lf))


"""
Explanation: Event Attribute Values

filter_event_attribute_values(log, attribute_key, values, level="case", retain=True)
retains (or drops) traces (or events) based on a given collection of values that need to be matched for the
  given attribute_key. If level=='case', complete traces are matched (or dropped if retain==False) that
  have at least one event that describes a specifeid value for the given attribute. If level=='event', only events
  that match are retained (or dropped).
End of explanation
"""


