How to improve identification of outliers for removal
You can try a quantile-quantile plot: you fit your data to a given distribution and plot your empirical distribution quantile against the theoretical quantiles. 

This can help you infer outliers in your sample as outliers for the fitted distribution.