Percentile as a threshold for Anomaly Detection? I'm following this article about Unsupervised Anomaly Detection Algorithms. In this article, a threshold value is calculated using the scipy score percentile method to determine whether the point is an outlier or not. What is the connection between percentile value and threshold and how can we decide if a point is an outlier or not using a threshold value?
You have a distribution of data (multidimensional or not it does not matter). And anomalies will be scattered towards corners of this dataset. Hence low percentile will indicate that certain points are anomalies in the data---cornered in. Threshold is, in this context, just a cutt-off value when you claim something is anomaly.