How to cluster/identify points away from a regression line
As already mentioned in a comment, there are some methods to detect outliers. The best known might be Cook‘s distance in (or after) regression. In essence you look for observations with a high residual.

Here is a short R tutorial. Here is a short Python approach.