"""
# **A/B TESTING**
 
 **What is A/B Testing?**

The A/B test is a hypothesis test in which two-sample user experiences are tested. In other words, A/B testing is a way to compare two versions of a single variable, typically by testing a subject's response to variant A against variant B, and determining which of the two variants is more effective.


"""
from PIL import Image 
Image.open("../input/ab-testing-pic/ab-testing-picc.png")
"""
 **A/B Testing with Business Problem**

***Business Problem***

The company recently introduced a new bid type, average bidding,
as an alternative to the current type of bidding called maximum bidding.

One of our client decided to test this new feature and wants to do
an A/B test to see if average bidding brings more returns than maximum bidding.

***The Story of the Data Set***

There are two separate data sets, the Control and the Test group.

***Variables***
* Impression: Number of ad views
* Click: Number of clicks on the displayed ad
* Purchase: The number of products purchased after clicked ads
* Earning: Earnings after the purchased products
"""
"""
**Required Modules and Libraries**
"""
!pip install openpyxl
import itertools
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import statsmodels.stats.api as sms
from scipy.stats import ttest_1samp, shapiro, levene, ttest_ind   
from statsmodels.stats.proportion import proportions_ztest

pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', 10)
pd.set_option('display.float_format', lambda x: '%.5f' % x)
df_c = pd.read_excel("../input/ab-testing/ab_testing.xlsx",sheet_name="Control Group")
df_t = pd.read_excel("../input/ab-testing/ab_testing.xlsx",sheet_name="Test Group")
"""
**Defining the hypothesis of the A/B test**

H0: There is no statistically significant difference the returns
 of the Maximum Bidding (control) and Average Bidding (test) options.
 
H1: There is statistically significant difference the returns
 of the Maximum Bidding (control) and Average Bidding (test) options.
 
 After the hypotheses are defined, the normality assumption and variance
 homogeneity are checked.In this direction, 
 the normality assumption control is performed first.
"""
df_c["Purchase"].mean()      #Let's observe the purchase mean of control group.

df_t["Purchase"].mean()      #Let's observe the purchase mean of test group.
data=[df_c["Purchase"],df_t["Purchase"]]
plt.boxplot(data);           #Let's see the boxplot grafic of purchases mean.
#Let's see the histogram grafic of purchases for both control and test group.
plt.figure(figsize=[10,5])
n, bins, patches = plt.hist(x=df_c["Purchase"], bins=10, color='#5F9EA0')
plt.xlabel('Purchase',fontsize=15)
plt.ylabel('Frequency',fontsize=15)
plt.title('Purchase of Control Group',fontsize=15)
plt.show()
plt.figure(figsize=[10,5])
n, bins, patches = plt.hist(x=df_t["Purchase"], bins=10, color='#3D59AB')
plt.xlabel('Purchase',fontsize=15)
plt.ylabel('Frequency',fontsize=15)
plt.title('Purchase of Test Group',fontsize=15)
plt.show()
"""
**Normality Assumption Control**

H0: The assumption of normality is provided.

H1: The assumption of normality is not provided.
"""

test_stat, pvalue = shapiro(df_c["Purchase"])
print('Test Stat = %.4f, p-value = %.4f' % (test_stat, pvalue))
#Test Stat = 0.9773, p-value = 0.5891

test_stat, pvalue = shapiro(df_t["Purchase"])
print('Test Stat = %.4f, p-value = %.4f' % (test_stat, pvalue))
#TTest Stat = 0.9589, p-value = 0.1541
"""
 p-value is less than 0.05, H0 is rejected.
 
 p-value is not less than 0.05, H0 can not rejected.
 
 So in this case, H0 can not rejected and the assumption of normality is provided.
"""
"""
**Variance Homogeneity Control**

 H0: The variances are homogeneous.
 
 H1: The variances are not homogeneous.
"""
test_stat, pvalue = levene(df_t["Purchase"],
                           df_c["Purchase"])

print('Test Stat = %.4f, p-value = %.4f' % (test_stat, pvalue))
#Test Stat = 2.6393, p-value = 0.1083
"""
 So in this case, H0 can not rejected and the variances are homogeneous.
"""
"""
**Because of the assumptions provided, non parametric test called two-sample t-test will be used.**
"""
test_stat, pvalue = ttest_ind(df_t["Purchase"],
                              df_c["Purchase"],
                              equal_var=True)

print('Test Stat = %.4f, p-value = %.4f' % (test_stat, pvalue))
# Test Stat = 0.9416, p-value = 0.3493
"""
So at the end, H0 can not rejected. We can infer that there is no statistically

significant difference the returns of the Maximum Bidding (control) and

Average Bidding (test) options.
"""
"""
**Using Two-Sample Rate Test**

H0: There is no statistically significant difference between the maximum bidding click-through rate and the average

bidding click-through rate.

H1: There is statistically significant difference between the maximum bidding click-through rate and the average 

bidding click-through rate
"""
Maks_succ_count=df_c["Click"].sum()     #204026
Ave_succ_count=df_t["Click"].sum()      #158701

Maks_rev_count=df_c["Impression"].sum()  #4068457
Ave_rev_count=df_t["Impression"].sum()   #4820496


test_stat, pvalue = proportions_ztest(count=[Maks_succ_count, Ave_succ_count],
                                      nobs= [df_c["Impression"].sum(),
                                            df_t["Impression"].sum()])
print('Test Stat = %.4f, p-value = %.4f' % (test_stat, pvalue))
"""
Test Stat = 129.3305, p-value = 0.0000

p-value is less than 0.05, H0 is rejected.According to Two-Sample Rate Test, There is statistically

significant difference between the maximum bidding click-through rate and the average bidding click-through rate
"""
"""

Consequently, two sample t-test were used primarily to see if there was a significant difference between the returns between the two groups. And it was concluded that there was no significant difference between the two groups. However, in this problem, two sample rate test were applied on request via ad viewing and click-through rates. As a result of the two sample rate test,there was a statistically significant difference between the click-through and ad viewing rates. So, we can say that different results can be obtained as a result of the tests used by considering different purposes and different variables.Therefore, it is necessary first of all to express the purpose in the best way and select the appropriate test methods.
"""