"""
CREDITABILITY (noun):

Appearance of truth or authenticity:
believability, color, credibility, credibleness, creditableness, plausibility, plausibleness, verisimilitude.

CREDIBILITY

Credibility comprises the objective and subjective components of the believability of a source or message. Credibility dates back to Aristotle theory of Rhetoric. Aristotle defines rhetoric as the ability to see what is possibly persuasive in every situation. 

He divided the means of persuasion into three categories, namely Ethos (the source's credibility), Pathos (the emotional or motivational appeals), and Logos (the logic used to support a claim), which he believed have the capacity to influence the receiver of a message. According to Aristotle, the term “Ethos” deals with the character of the speaker. The intent of the speaker is to appear credible.

In fact, the speaker's ethos is a rhetorical strategy employed by an orator whose purpose is to "INSPIRE TRUST IN HIS AUDIENCE.” Credibility has two key components: trustworthiness and expertise, which both have objective and subjective components. 

Trustworthiness is based more on subjective factors, but can include objective measurements such as established reliability. Expertise can be similarly subjectively perceived, but also includes relatively objective characteristics of the source or message (e.g., credentials, certification or information quality).

Secondary components of credibility include source dynamism (charisma) and physical attractiveness.

https://en.wikipedia.org/wiki/Credibility
"""
"""
#Even Birds know the Meaning of Creditability

![](https://thesaurus.plus/img/synonyms/132/creditability.png)thesaurusplus
"""
# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
import warnings
warnings.filterwarnings("ignore")

# Input data files are available in the read-only "../input/" directory
# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory

import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using "Save & Run All" 
# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session
"""
<font color="#EC7063">ETHOS. The source's credibility. </font>

### <b><mark style="background-color: #9B59B6"><font color="white">* Has the applicant sources to pay the loan?   *</font></mark></b>

<font color="#EC7063">PATHOS. The emotional or motivational appeals. </font>

### <b><mark style="background-color: #9B59B6"><font color="white">* Has the applicant good motivations to require the loan?   *</font></mark></b>

<font color="#EC7063">LOGOS. The logic used to support a claim. </font>

### <b><mark style="background-color: #9B59B6"><font color="white">* Has the applicant convinced the bank that the loan will be paid?   *</font></mark></b>
"""
nRowsRead = 1000 # specify 'None' if want to read whole file
dataset = pd.read_csv('../input/cusersmarildownloadsgermancsv/german.csv', delimiter=';', encoding = "ISO-8859-2", nrows = nRowsRead)
dataset.dataframeName = 'german.csv'
nRow, nCol = dataset.shape
print(f'There are {nRow} rows and {nCol} columns')
dataset.head()
"""
The predictors that may potentially have any influence on Creditability:

Account Balance: No account (1), None (No balance) (2), Some Balance (3)

Payment Status: Some Problems (1), Paid Up (2), No Problems (in this bank) (3)

Savings/Stock Value: None, Below 100 DM, (100, 1000) DM, Above 1000 DM

Employment Length: Below 1 year (including unemployed), (1, 4), (4, 7), Above 7

Sex/Marital Status: Male Divorced/Single, Male Married/Widowed, Female

No of Credits at this bank: 1, More than 1

Guarantor: None, Yes

Concurrent Credits: Other Banks or Dept Stores, None

ForeignWorker variable may be dropped from the study

Purpose of Credit: New car, Used car, Home Related, Other
"""
"""
#All script by Baris Cal https://www.kaggle.com/bariscal/portugal-wine-dataset-quality-prediction-w-ml 
"""
from sklearn import tree
import graphviz 
import os
import preprocessing 

from pandas_profiling import ProfileReport

from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import chi2, f_classif
from sklearn.model_selection import KFold
from sklearn.feature_selection import SelectFromModel
from sklearn.svm import LinearSVC

from sklearn.model_selection import cross_val_score
from sklearn.model_selection import cross_val_predict

from sklearn.preprocessing import normalize
from sklearn.preprocessing import StandardScaler, LabelEncoder, OneHotEncoder
from sklearn.model_selection import train_test_split

from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.decomposition import PCA
from sklearn.naive_bayes import GaussianNB
from sklearn.naive_bayes import MultinomialNB
from sklearn.naive_bayes import BernoulliNB
from sklearn.naive_bayes import CategoricalNB
from sklearn.linear_model import LinearRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC
from sklearn.cluster import KMeans
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.preprocessing import StandardScaler
from xgboost import XGBClassifier

from sklearn.metrics import confusion_matrix, accuracy_score
from sklearn.metrics import precision_score, recall_score, f1_score
from sklearn.metrics import classification_report
from sklearn.model_selection import GridSearchCV, cross_val_score
from sklearn.model_selection import GridSearchCV


from sklearn.linear_model import SGDClassifier, LogisticRegression


from sklearn.neural_network import MLPClassifier
from xgboost import XGBClassifier, XGBRFClassifier
from xgboost import plot_tree, plot_importance

from sklearn.metrics import confusion_matrix, accuracy_score, roc_auc_score, roc_curve
from sklearn import preprocessing
from sklearn.feature_selection import RFE


from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
list(dataset.columns)
features = ['Account_Balance',
 'Duration_of_Credit_monthly',
 'Payment_Status_of_Previous_Credit',
 'Purpose',
 'Credit_Amount',
 'Value_Savings_Stocks',
 'Guarantors',
 'Most_valuable_available_asset',
 'Age_years',
 'No_of_Credits_at_this_Bank',
 'Occupation']
sns.set_style('darkgrid')
#sns.pairplot(dataset, hue = 'Creditability')
mask = np.zeros_like(dataset[features].corr(), dtype=np.bool) 
mask[np.triu_indices_from(mask)] = True 

f, ax = plt.subplots(figsize=(16, 12))
plt.title('Pearson Correlation Matrix',fontsize=25)

sns.set_style('darkgrid')
sns.heatmap(dataset[features].corr(),linewidths=0.25,vmax=0.7,square=True,cmap="BuGn", #"BuGn_r" to reverse 
            linecolor='w',annot=True,annot_kws={"size":8},mask=mask,cbar_kws={"shrink": .9});
plt.figure(figsize=(12,8)) 
sns.heatmap(dataset.corr(), annot=True, cmap='Dark2_r', linewidths = 2)
plt.show()
plt.figure(figsize=(30,30))

plt.subplot(4,4,1)
sns.distplot(dataset['Creditability']).set_title('Creditability Interval')

plt.subplot(4,4,2)
sns.distplot(dataset['Account_Balance']).set_title('Account Balance Interval')

plt.subplot(4,4,3)
sns.distplot(dataset['Purpose']).set_title('Purpose Interval')

plt.subplot(4,4,4)
sns.distplot(dataset['Guarantors']).set_title('Guarantors Interval')

plt.subplot(4,4,5)
sns.distplot(dataset['Most_valuable_available_asset']).set_title('Most valuable available asset Interval')

plt.subplot(4,4,6)
sns.distplot(dataset['Age_years']).set_title('Age Interval')

plt.subplot(4,4,7)
sns.distplot(dataset['Occupation']).set_title('Occupation Interval')

plt.subplot(4,4,8)
sns.distplot(dataset['Value_Savings_Stocks']).set_title('Value Savings & Stocks Interval')
plt.figure(figsize=(20,15))
plt.subplot(2,2,1)
sns.violinplot(x = 'Creditability', y = 'Account_Balance', data = dataset)
plt.subplot(2,2,2)
sns.violinplot(x = 'Creditability', y = 'Credit_Amount', data = dataset)
plt.subplot(2,2,3)
sns.violinplot(x = 'Creditability', y = 'Occupation', data = dataset)
plt.subplot(2,2,4)
sns.violinplot(x = 'Creditability', y = 'Guarantors', data = dataset)
"""
#Apple co-founder says Apple Card algorithm gave wife LOWER CREDIT LIMIT

Apple Inc AAPL.O co-founder Steve Wozniak joined in the online debate over accusations of GENDER DISCRIMINATION by the ALGORITHM behind the iPhone maker's credit card, fueling scrutiny of the newly launched Apple Card.

The criticism started after entrepreneur David Heinemeier Hansson railed against the Apple Card in a series of Twitter posts, saying it gave him 20 times the credit limit his wife received.

Apple Card applicants were evaluated independently, according to income and creditworthiness, taking into account factors such as personal credit scores and personal debt.

Wozniak said he got 10 times more credit on the card, compared with his wife.

https://www.reuters.com/article/us-goldman-sachs-apple/apple-co-founder-says-apple-card-algorithm-gave-wife-lower-credit-limit-idUKKBN1XL038
"""
dataset["Sex_Marital_Status"].value_counts()
plt.figure(1, figsize=(10,10))
dataset['Sex_Marital_Status'].value_counts().plot.pie(autopct="%1.1f%%")
plt.title ('Sex & Marital Status')
"""
#Sex/Marital Status: Male Divorced/Single, Male Married/Widowed, Female.

#They didn't mentioned which number corresponds to the status. What percent correspond to females if I don't know female number?

https://online.stat.psu.edu/stat508/resources/analysis/gcd/gcd.1
"""
features = ['Account_Balance',
 'Duration_of_Credit_monthly',
 'Payment_Status_of_Previous_Credit',
 'Purpose',
 'Credit_Amount',
 'Value_Savings_Stocks',
 'Guarantors',
 'Most_valuable_available_asset',
 'Age_years',
 'No_of_Credits_at_this_Bank',
 'Occupation']
label = ['Creditability']

X = dataset[features]
y = dataset[label]
"""
#Attention Below it's NOT 12 it's l2 (lower case L)
"""
X = normalize(X, norm = 'l2')
print(X[:3])
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=101) 
X_valid, X_test, y_valid, y_test = train_test_split(X_test, y_test, test_size=0.5, random_state=42)

print(f'Total # of sample in whole dataset: {len(X)}')
print(f'Total # of sample in train dataset: {len(X_train)}')
print(f'Total # of sample in validation dataset: {len(X_valid)}')
print(f'Total # of sample in test dataset: {len(X_test)}')
"""
#Score of Models
"""
#Scores of Models
models = {
    'GaussianNB': GaussianNB(),
    'MultinomialNB': MultinomialNB(),
    'BernoulliNB': BernoulliNB(),
    'LogisticRegression': LogisticRegression(),
    'RandomForestClassifier': RandomForestClassifier(),
    'SupportVectorMachine': SVC(),
    'DecisionTreeClassifier': DecisionTreeClassifier(),
    'KNeighborsClassifier': KNeighborsClassifier(),
    'GradientBoostingClassifier': GradientBoostingClassifier(),
    'Stochastic Gradient Descent':  SGDClassifier(max_iter=5000, random_state=0),
    'Neural Nets': MLPClassifier(solver='lbfgs', alpha=1e-5, hidden_layer_sizes=(5000, 10), random_state=1),
    'XGBClassifier': XGBClassifier()
}

modelNames = ["GaussianNB","MultinomialNB",'BernoulliNB','LogisticRegression','RandomForestClassifier','SupportVectorMachine',
             'DecisionTreeClassifier', 'KNeighborsClassifier','GradientBoostingClassifier',
             'Stochastic Gradient Descent', 'Neural Nets', 'XGBClassifier']

trainScores = []
validationScores = []
testScores = []

for m in models:
  model = models[m]
  model.fit(X_train, y_train)
  score = model.score(X_valid, y_valid)
  #print(f'{m} validation score => {score*100}')
    
  print(f'{m}') 
  train_score = model.score(X_train, y_train)
  print(f'Train score of trained model: {train_score*100}')
  trainScores.append(train_score*100)

  validation_score = model.score(X_valid, y_valid)
  print(f'Validation score of trained model: {validation_score*100}')
  validationScores.append(validation_score*100)

  test_score = model.score(X_test, y_test)
  print(f'Test score of trained model: {test_score*100}')
  testScores.append(test_score*100)
  print(" ")
    
  y_predictions = model.predict(X_test)
  conf_matrix = confusion_matrix(y_predictions, y_test)

  print(f'Confussion Matrix: \n{conf_matrix}\n')

  predictions = model.predict(X_test)
  cm = confusion_matrix(predictions, y_test)

  tn = conf_matrix[0,0]
  fp = conf_matrix[0,1]
  tp = conf_matrix[1,1]
  fn = conf_matrix[1,0]
  accuracy  = (tp + tn) / (tp + fp + tn + fn)
  precision = tp / (tp + fp)
  recall    = tp / (tp + fn)
  f1score  = 2 * precision * recall / (precision + recall)
  specificity = tn / (tn + fp)
  print(f'Accuracy : {accuracy}')
  print(f'Precision: {precision}')
  print(f'Recall   : {recall}')
  print(f'F1 score : {f1score}')
  print(f'Specificity : {specificity}')
  print("") 
  print(f'Classification Report: \n{classification_report(predictions, y_test)}\n')
  print("")
   
  for m in range (1):
    current = modelNames[m]
    modelNames.remove(modelNames[m])

  preds = model.predict(X_test)
  confusion_matr = confusion_matrix(y_test, preds) #normalize = 'true'
  print("############################################################################")
  print("")
  print("")
  print("")
plt.figure(figsize=(20,10))
plt.title('Train - Validation - Test Scores of Models', fontweight='bold', size = 24)

barWidth = 0.25
 
bars1 = trainScores
bars2 = validationScores
bars3 = testScores
 
r1 = np.arange(len(bars1))
r2 = [x + barWidth for x in r1]
r3 = [x + barWidth for x in r2]
 
plt.bar(r1, bars1, color='blue', width=barWidth, edgecolor='white', label='train', yerr=0.5,ecolor="black",capsize=10)
plt.bar(r2, bars2, color='#557f2d', width=barWidth, edgecolor='white', label='validation', yerr=0.5,ecolor="black",capsize=10, alpha = .50)
plt.bar(r3, bars3, color='red', width=barWidth, edgecolor='white', label='test', yerr=0.5,ecolor="black",capsize=10, hatch = '-')
 
modelNames = ["GaussianNB","MultinomialNB",'BernoulliNB','LogisticRegression','RandomForestClassifier','SupportVectorMachine',
             'DecisionTreeClassifier', 'KNeighborsClassifier','GradientBoostingClassifier',
             'Stochastic Gradient Descent', 'Neural Nets', 'XGBClassifier']
    
plt.xlabel('Algorithms', fontweight='bold', size = 24)
plt.ylabel('Scores', fontweight='bold', size = 24)
plt.xticks([r + barWidth for r in range(len(bars1))], modelNames, rotation = 75)
 
plt.legend()
plt.show()
for i in range(12):
    print(f'Accuracy of {modelNames[i]} -----> {testScores[i]}')
"""
The best result was from XGBClassifier with 75.15 accuracy score.

That phrase below I simply didn't get. Maybe I'll ask Baris Cal.

"We had 7 labels to predict. Now, i will decrease this 7 labels to 3 and try to increase accuracy scores."
"""
dataset['Creditability'].value_counts()
"""
Improving Results

Dataset includes 'Creditability' attribute. This atribute values are between 0 and 1. So I think I don't even have to decrease the value interval.



On the original Dataset (Portugal Wines) the attribute "quality" values are between 0 and 9

"For increase the accuracy of results, I have done decrease the value interval. So I created an attribute. According to these attribute, if 'quality' value of sample is lower than 5, it will be classified as 'low', if 6 or 7 it will be in 'medium' range and the others will be 'high' quality wines. For use this new data in machine learning algorithms, I made this values integers. 'low' quality is 0, 'medium' quality is 1 and 'high' quality is 2. I not gonna use 'quality_value' attribute in my application. I did it to see how did I obtain that values."
https://www.kaggle.com/bariscal/portugal-wine-dataset-quality-prediction-w-ml
"""
#Code by Olga Belitskaya https://www.kaggle.com/olgabelitskaya/sequential-data/comments
from IPython.display import display,HTML
c1,c2,f1,f2,fs1,fs2=\
'#eb3434','#eb3446','Akronim','Smokum',30,15
def dhtml(string,fontcolor=c1,font=f1,fontsize=fs1):
    display(HTML("""<style>
    @import 'https://fonts.googleapis.com/css?family="""\
    +font+"""&effect=3d-float';</style>
    <h1 class='font-effect-3d-float' style='font-family:"""+\
    font+"""; color:"""+fontcolor+"""; font-size:"""+\
    str(fontsize)+"""px;'>%s</h1>"""%string))
    
    
dhtml('Be patient. Marília Prata, @mpwolke was Here')