"""
Hellooooo, I ❤️ VBO
- I will explain my Association Rule Learning Recommender project in my weekly assignments in order to turn it into a contribution for you.
"""
"""
![image.png](attachment:ee702f43-6a1e-4730-83b6-7240540e9c45.png)
"""
"""
# **Association Rule Learning Recommender**
"""
"""
# Association Rules

Association rule mining is a methodology that is used to discover unknown relationships hidden in big data. Rules refer to a set of identified frequent itemsets that represent the uncovered relationships in the dataset. The underlying idea is to identify rules that will predict the occurence of one or more items based on the occurrences of other items in the dataset. It is an unsupervised machine learning method. This means that no direct guiding output data is given to find the patterns.

In many commercial environments large quantities of data is accumulated in databases from day-to-day operations. This lays the foundation for mining association rules. In retail, for example, customer purchase data are collected on a daily basis at the checkout counters of city stores or when shopping at online stores. The accumulated data items are often market basket transactions. Managers of stores are interested in analyzing the collected data in order to learn the purchasing behaviour of customers. This enables a large variety of business-related applications (see below) based on the identified rules in the data.

Below is one of the famous examples illustrating a rule based on a strong relationship between the sale of Diapers and the sale of Beer. In other words many customers who buy Diapers also buy Beer. Investigating the transactions below in order to find those frequent itemsets seems to be easy. But consider in real datasets millions or billions of transactions are searched across 100000 of different items that may identify 1000 of rules. This motivates the automation of the process using association rule mining algorithms. In retail these rules help to identify new opportunities and ways for cross-selling products to customers.

![image.png](attachment:6ae1bfdc-4419-45d2-a498-b209ec260aef.png)


The example above illustrated the core idea of association rule mining based on frequent itemsets. It is a simple example since it essentially ignores other relevant attributes such as quantity of items sold, the Price, or even a specific brand of items. The mining process to discover unknown patterns in large transaction datasets is computationally expensive. A particular challenge is also the size of the millions or billions of the transaction dataset when performing mining within memory. In other words smart out-of-cpu strategies are needed to work on the big datasets as a whole in order to identify frequent itemsets. This is the case since sub-sampling of dataset items may increase the risk to overlook frequent itemset patterns.

Another challenge when performing association rule mining is that some uncovered patterns are simply not true since they have happen by chance. The latter therefore needs often manual post-processing or even application domain knowledge in order to find true rules out of the undiscovered patterns. Important is to find actionable insights from rules that can be used to change a product portfolio, store setup, or customer relations. Several configuration options are available for association rules (e.g. support, confidence) in order to provide a more clear set of rules.
"""
"""
# Association Rules Applications

Understanding the customer purchasing behaviour by using association rule mining enables different applications. As shown above rules help to identify new opportunities and ways for cross-selling products to customers. It is used for personalised marketing promotions, smarter inventory management, product placement strategies in stores, and a better customer relationship management.
"""
"""
# Business Problem:

Suggesting products to users at the basket stage.
"""
"""
# Dataset Story

* The dataset named Online Retail II shows the sales of a UK-based online store between 01/12/2009 - 09/12/2011.
contains.

* The product catalog of this company includes souvenirs. promotion
can be considered as products.

* There is also information that most of its customers are wholesalers.
"""
"""
# Variables

* InvoiceNo – Invoice Number

> If this code starts with C, it means that the operation has been cancelled.

* StockCode – Product code

> Unique number for each product.

* Description – Product name

* Quantity – Number of products

> It expresses how many of the products on the invoices have been sold.

* InvoiceDate – Invoice date

* UnitPrice – Invoice price (Sterling)

* CustomerID – Unique customer number

* Country – Country name
"""
"""
# Project Steps
"""
"""
# TASK 1: 

Perform Data Preprocessing

**Important note!**
*Select the 2010-2011 data and preprocess all the data.
The choice of Germany will be in the next step.*
"""
import pandas as pd
pd.set_option("display.max_columns", None)
pd.set_option("display.width", 500)
pd.set_option("display.expand_frame_repr", False)
from mlxtend.frequent_patterns import apriori, association_rules
!pip install xlrd
!pip install openpyxl
df_ = pd.read_excel("../input/uci-online-retail-ii-data-set/online_retail_II.xlsx", sheet_name= "Year 2010-2011")
df = df_.copy()
df.head()
# To avoid the outliers
def outlier_thresholds(dataframe, variable):
    quartile1 = dataframe[variable].quantile(0.01)
    quartile3 = dataframe[variable].quantile(0.99)
    interquantile_range = quartile3 - quartile1
    up_limit = quartile3 + 1.5 * interquantile_range
    low_limit = quartile1 - 1.5 * interquantile_range
    return low_limit, up_limit
def replace_with_thresholds(dataframe, variable):
    low_limit, up_limit = outlier_thresholds(dataframe, variable)
    dataframe.loc[(dataframe[variable] < low_limit), variable] = low_limit
    dataframe.loc[(dataframe[variable] > up_limit), variable] = up_limit
# Preprocessing
def retail_data_prep(dataframe):
    dataframe.dropna(inplace=True)
    dataframe = dataframe[~dataframe["Invoice"].str.contains("C", na=False)]
    dataframe = dataframe[dataframe["Quantity"] > 0]
    dataframe = dataframe[dataframe["Price"] > 0]
    replace_with_thresholds(dataframe, "Quantity")
    replace_with_thresholds(dataframe, "Price")
    return dataframe
df_germany = df[df["Country"] == "Germany"]
df_germany.groupby(["Invoice","Description"]).agg({"Quantity":"sum"}).head(20)
df_germany.groupby(['Invoice', 'Description']).agg({"Quantity": "sum"}).unstack().iloc[0:5, 0:5]
df_germany.groupby(['Invoice', 'Description']).agg({"Quantity": "sum"}).unstack().fillna(0).iloc[0:5, 0:5]
df_germany.groupby(['Invoice', 'Description']).agg({"Quantity": "sum"}).unstack().fillna(0).applymap(lambda x: 1 if x > 0 else 0).iloc[0:5, 0:5]
"""
# TASK 2: 

Generate association rules through Germany customers.
"""
def create_invoice_product_df(dataframe, id=False):
    if id:
        return dataframe.groupby(["Invoice", "StockCode"])["Quantity"].sum().unstack().fillna(0). \
            applymap(lambda x: 1 if x > 0 else 0)
    else:
        return dataframe.groupby(["Invoice", "Description"])["Quantity"].sum().unstack().fillna(0). \
            applymap(lambda x: 1 if x > 0 else 0)
ger_inv_pro_df = create_invoice_product_df(df_germany)
ger_inv_pro_df = create_invoice_product_df(df_germany, id=True)
# For the find the names of id
def check_id(dataframe, stock_code):
    product_name = dataframe[dataframe["StockCode"] == stock_code][["Description"]].values[0].tolist()
    print(product_name)
"""
# TASK 3: 

What are the names of the products whose IDs are given?
"""
check_id(df_germany, 21987) # ['PACK OF 6 SKULL PAPER CUPS']
check_id(df_germany, 23235) # ['STORAGE TIN VINTAGE LEAF']
check_id(df_germany, 22747) # ["POPPY'S PLAYHOUSE BATHROOM"]
"""
# TASK 4: 

Make a product recommendation for the users in the cart.
"""
frequent_itemsets = apriori(ger_inv_pro_df, min_support=0.01, use_colnames=True)
frequent_itemsets.sort_values("support", ascending=False).head(20)
rules = association_rules(frequent_itemsets, metric="support", min_threshold=0.01)
rules.sort_values("support", ascending=False).head(100)
rules.sort_values("lift", ascending=False).head(100)
# Product recommendation:
def arl_recommender(rules_df, product_id, rec_count=1):
    sorted_rules = rules_df.sort_values("lift", ascending=False)
    recommendation_list = []
    for i, product in enumerate(sorted_rules["antecedents"]):
        for j in list(product):
            if j == product_id:
                recommendation_list.append(list(sorted_rules.iloc[i]["consequents"])[0])

    return recommendation_list[0:rec_count]
arl_recommender(rules, 21987,2) # [21086, 21989]
check_id(df_germany, 21086) # ['SET/6 RED SPOTTY PAPER CUPS']
arl_recommender(rules, 22747,1)# [22746]
check_id(df_germany, 22746) # ["POPPY'S PLAYHOUSE LIVINGROOM "]
"""
# <span style="font-family:cursive;"> Thank youu:)))) </span>
"""