# ---
# jupyter:
#   jupytext:
#     text_representation:
#       extension: .py
#       format_name: light
#       format_version: '1.5'
#       jupytext_version: 1.14.4
#   kernelspec:
#     display_name: Python 3
#     language: python
#     name: python3
# ---

# ### Author : <NAME>
# ### Topic : All about Feature Engineering 
# ### Email : <EMAIL>

# # Section : Feature Selection 

# ### What is Feature Selection Techniques
# Feature Selection is one of the core concepts in machine learning which hugely impacts the performance of your model. The data features that you use to train your machine learning models have a huge influence on the performance you can achieve.
# Irrelevant or partially relevant features can negatively impact model performance.

# ### Benefits of performing feature selection before modeling your data?
#
# <b>Reduces Overfitting:</b> Less redundant data means less opportunity to make decisions based on noise.
#
# <b>Improves Accuracy:</b> Less misleading data means modeling accuracy improves.
#
# <b>Reduces Training Time:</b> fewer data points reduce algorithm complexity and algorithms train faster.

# ### Feature Selection Methods:
# 1. Univariate Selection
# 2. Feature Importance
# 3. Correlation Matrix with Heatmap

# ### 1. Univariate Selection
# Some Function:
#     1. chi-square : Best selection for categorial data
#     2. f_classif
#     3. mutual_info_classif

# ### Import Necessary Libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# ### Import Dataset

df = pd.read_csv(r'F:\Feature Engineering\Univariate Selection\cardio_train.csv',sep = ";")
df

df.shape

df['yr'] = (df['age']/365).round(0)

df['yr']

df.describe()

df.corr()

df = df.drop(['yr'],axis = 1)

x = df.iloc[:,:-1]

y = df.iloc[:,12]

y

from sklearn.feature_selection import SelectKBest

from sklearn.feature_selection import f_classif

FIT_FEATURES = SelectKBest(score_func = f_classif)

FIT_FEATURES.fit(x,y)

pd.DataFrame(FIT_FEATURES.scores_)

SCORE_COL = pd.DataFrame(FIT_FEATURES.scores_,columns = ['score value'])

SCORE_COL

NAME_COL = pd.DataFrame(x.columns)

pd.DataFrame(NAME_COL)

topr_features = pd.concat([NAME_COL,SCORE_COL], axis =1)

topr_features

topr_features.nlargest(8,'score value')



from sklearn.model_selection import train_test_split

xtrain,xtest,ytrain,ytest = train_test_split(x,y,test_size=.3,random_state = 1)

xtrain

from sklearn.ensemble import RandomForestClassifier

rfc = RandomForestClassifier()

rfc.fit(xtrain,ytrain)

rfc.score(xtest,ytest)

# #  Section : Feature Importance

from sklearn.ensemble import ExtraTreesClassifier

model = ExtraTreesClassifier()

model.fit(x,y)

model.feature_importances_

top = pd.Series(model.feature_importances_,index = x.columns)

top

# #### Graph Show

top.nlargest(10).plot(kind = 'barh')

# # Section : Count Vectorizer

# Scikit-learn’s CountVectorizer is used to transform a corpora of text to a vector of term / token counts. It also provides the capability to preprocess your text data prior to generating the vector representation making it a highly flexible feature representation module for text.

# #### Methods
# build_analyzer():Return a callable that handles preprocessing, tokenization and n-grams generation.
#
# build_preprocessor():Return a function to preprocess the text before tokenization.
#
# build_tokenizer():Return a function that splits a string into a sequence of tokens.
#
# decode(doc):Decode the input into a string of unicode symbols.
#
# fit(raw_documents[, y]):Learn a vocabulary dictionary of all tokens in the raw documents.
#
# fit_transform(raw_documents[, y]):Learn the vocabulary dictionary and return document-term matrix.
#
# get_feature_names():Array mapping from feature integer indices to feature name.
#
# get_params([deep]):Get parameters for this estimator.
#
# get_stop_words():Build or fetch the effective stop words list.
#
# inverse_transform(X):Return terms per document with nonzero entries in X.
#
# set_params(**params):Set the parameters of this estimator.
#
# transform(raw_documents):Transform documents to document-term matrix.

# #### Word Counts with CountVectorizer
# The CountVectorizer provides a simple way to both tokenize a collection of text documents and build a vocabulary of known words, but also to encode new documents using that vocabulary.
#
# You can use it as follows:
#
# Create an instance of the CountVectorizer class.
# Call the fit() function in order to learn a vocabulary from one or more documents.
# Call the transform() function on one or more documents as needed to encode each as a vector.
# An encoded vector is returned with a length of the entire vocabulary and an integer count for the number of times each word appeared in the document.
#
# Because these vectors will contain a lot of zeros, we call them sparse. Python provides an efficient way of handling sparse vectors in the scipy.sparse package.
#
# The vectors returned from a call to transform() will be sparse vectors, and you can transform them back to numpy arrays to look and better understand what is going on by calling the toarray() function.

# ### Import Libraries

from sklearn.feature_extraction.text import CountVectorizer

# ### Dataset

dataset = ['Hey welcome to datascience',
          'This is Data Science Course',
          'Working as data scientist']

dataset

# ### Apply Count vectorizer

cv = CountVectorizer()
x = cv.fit_transform(dataset)

cv.get_feature_names()

x.toarray()

# # Section : TF - IDF Vectorizer

# ### What is TF-IDF
# TF-IDF stands for Term Frequency — Inverse Document Frequency and is a statistic that aims to better define how important a word is for a document, while also taking into account the relation to other documents from the same corpus.
#
# This is performed by looking at how many times a word appears into a document while also paying attention to how many times the same word appears in other documents in the corpus.
#
# The rationale behind this is the following: a word that frequently appears in a document has more relevancy for that document, meaning that there is higher probability that the document is about or in relation to that specific word.
#
# a word that frequently appears in more documents may prevent us from finding the right document in a collection; the word is relevant either for all documents or for none. Either way, it will not help us filter out a single document or a small subset of documents from the whole set.

# ### Import Libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

# ### Tf-IDF Vectorizer Library

from sklearn.feature_extraction.text import TfidfVectorizer

# ### DataSet

documentA = 'the man went out for a walk'
documentB = 'the children sat around the fire'

bagOfWordsA = documentA.split(' ')
bagOfWordsB = documentB.split(' ')

bagOfWordsA

bagOfWordsB

# ### Unique Words

uniqueWords = set(bagOfWordsA).union(set(bagOfWordsB))

numOfWordsA = dict.fromkeys(uniqueWords, 0)
for word in bagOfWordsA:
    numOfWordsA[word] += 1
numOfWordsB = dict.fromkeys(uniqueWords, 0)
for word in bagOfWordsB:
    numOfWordsB[word] += 1

numOfWordsA

numOfWordsB

# # Section :  Feature Scaling : Min-Max Scaler | Standardization

# ### Feature Scaling
# Feature Scaling is a technique to standardize the independent features present in the data in a fixed range. It is performed during the data pre-processing to handle highly varying magnitudes or values or units. If feature scaling is not done, then a machine learning algorithm tends to weigh greater values, higher and consider smaller values as the lower values, regardless of the unit of the values.
#
# Example: If an algorithm is not using feature scaling method then it can consider the value 3000 meter to be greater than 5 km but that’s actually not true and in this case, the algorithm will give wrong predictions. So, we use Feature Scaling to bring all values to same magnitudes and thus, tackle this issue.

# ### Example of Algorithm where Feature Scaling Matters :
# K-Means
# K-Nearest Neighbors
# Principle Component Analysis
# Gradient Desent Mainly Distance Base algorithms are affected by Feature Scaling
# Note: Naive Bayes, Linear Discriminant Analysis,Tree based algorithm are not affected by feature scaling

# ### The Big Question – Normalize or Standardize?
# Normalization vs. standardization is an eternal question among machine learning newcomers. Let me elaborate on the answer in this section.
#
# Normalization is good to use when you know that the distribution of your data does not follow a Gaussian distribution. This can be useful in algorithms that do not assume any distribution of the data like K-Nearest Neighbors and Neural Networks.
#
# Standardization, on the other hand, can be helpful in cases where the data follows a Gaussian distribution. However, this does not have to be necessarily true. Also, unlike normalization, standardization does not have a bounding range. So, even if you have outliers in your data, they will not be affected by standardization.

# ### Import Libraries

import numpy as np
import pandas as pd

# ### Import Dataset

data_set = pd.read_csv(r'F:\Feature Engineering\Feature Scaling\feature_scaling.csv')
data_set

x = data_set.iloc[:,1:3]

x

# ### MIN MAX SCALER

from sklearn.preprocessing import MinMaxScaler

mns = MinMaxScaler(feature_range = (0,1))

x_after_min_max_scaler = mns.fit_transform(x)

x_after_min_max_scaler

# ### Standardisation

from sklearn.preprocessing import StandardScaler

Standardisation = StandardScaler()

x_after_standardisation = Standardisation.fit_transform(x)

x_after_standardisation



StarcoderdataJupyterScriptsDedupFiltered