feature-engineering-and-data-transformations/using-toy-exmaples/feature_cleaning/outlier.py
def outlier_detect_mean_std(data,col,threshold=3):
    '''
    outlier detection by Mean and Standard Deviation Method.
    If a value is a certain number(called threshold) of standard deviations away 
    from the mean, that data point is identified as an outlier. 
    Default threshold is 3.

    This method can fail to detect outliers because the outliers increase the standard deviation. 
    The more extreme the outlier, the more the standard deviation is affected.
    '''
   
    Upper_fence = data[col].mean() + threshold * data[col].std()
    Lower_fence = data[col].mean() - threshold * data[col].std()   
    para = (Upper_fence, Lower_fence)   
    tmp = pd.concat([data[col]>Upper_fence,data[col]<Lower_fence],axis=1)
    outlier_index = tmp.any(axis=1)
    print('Num of outlier detected:',outlier_index.value_counts()[1])
    print('Proportion of outlier detected',outlier_index.value_counts()[1]/len(outlier_index))
    return outlier_index, para

    outlier detection by Mean and Standard Deviation Method.
    If a value is a certain number(called threshold) of standard deviations away 
    from the mean, that data point is identified as an outlier. 
    Default threshold is 3.

    This method can fail to detect outliers because the outliers increase the standard deviation. 
    The more extreme the outlier, the more the standard deviation is affected.
    
outlier detection by Mean and Standard Deviation Method.
If a value is a certain number(called threshold) of standard deviations away
from the mean, that data point is identified as an outlier.
Default threshold is 3.

This method can fail to detect outliers because the outliers increase the standard deviation.
The more extreme the outlier, the more the standard deviation is affected.
outlier detection by Mean and Standard Deviation Method . If a value is a certain number ( called threshold ) of standard deviations away from the mean that data point is identified as an outlier . Default threshold is 3 . This method can fail to detect outliers because the outliers increase the standard deviation . The more extreme the outlier the more the standard deviation is affected .
def outlier_detect_mean_std(data,col,threshold=3):
    Upper_fence = data[col].mean() + threshold * data[col].std()
    Lower_fence = data[col].mean() - threshold * data[col].std()   
    para = (Upper_fence, Lower_fence)   
    tmp = pd.concat([data[col]>Upper_fence,data[col]<Lower_fence],axis=1)
    outlier_index = tmp.any(axis=1)
    print('Num of outlier detected:',outlier_index.value_counts()[1])
    print('Proportion of outlier detected',outlier_index.value_counts()[1]/len(outlier_index))
    return outlier_index, para
def outlier_detect_mean_std ( data , col , threshold = 3 ) : Upper_fence = data [ col ] . mean ( ) + threshold * data [ col ] . std ( ) Lower_fence = data [ col ] . mean ( ) - threshold * data [ col ] . std ( ) para = ( Upper_fence , Lower_fence ) tmp = pd . concat ( [ data [ col ] > Upper_fence , data [ col ] < Lower_fence ] , axis = 1 ) outlier_index = tmp . any ( axis = 1 ) print ( 'Num of outlier detected:' , outlier_index . value_counts ( ) [ 1 ] ) print ( 'Proportion of outlier detected' , outlier_index . value_counts ( ) [ 1 ] / len ( outlier_index ) ) return outlier_index , para
outlier detection by Mean and Standard Deviation Method.
outlier detection by Mean and Standard Deviation Method .
'''
    outlier detection by Mean and Standard Deviation Method.
    If a value is a certain number(called threshold) of standard deviations away 
    from the mean, that data point is identified as an outlier. 
    Default threshold is 3.

    This method can fail to detect outliers because the outliers increase the standard deviation. 
    The more extreme the outlier, the more the standard deviation is affected.
    '''
data
col
threshold