Which Outlier Detection Method? Why? For detecting an outlier in a vector I have tested different well known outlier detection methods. Finally, I used combination of different methods and an agreement between those methods. Now, a person asks why did you choose this combination and algorithms!? You can reach different combinations and use other algorithms and they may yield better results. What should I answer? I cannot just say based on tests as there are many other algorithms that I haven't tested (cannot test all algorithms). It is not a logical response, I think. 

I'm looking for tests to justify my selected methods and combination and say why I have selected these methods. 

Please let me know your suggestions.
You can justify your choices by using data.

Treat the anomaly detection like a supervised learning problem where the concept is being anomaly.
Then you'll be able to present - for each method - its confusion matrix.
Not only it will be a good justification, it will enable to understand the expected results.

Many times, we have models and we wonder which confidence threshold we should use for alerting. In the supervised learning framework you'll be able to do trade-off like "increasing the confidence to X will lead to a better precision Y yet a decrease of the recall to Z".