What is the best statistical measure tool to measure how close data is to fitted regression line if outliers are not fitted
When you mention that your model does a good job in fitting the data (and not the outliers), I am assuming you are taking a look directly into actual vs predicted (or fitted) values. While doing this, you do not look for comparisons for outlier observations. 

Similarly, you could still calculate the R squared by removing the outlier observations from the calculation, and it could still serve as a measure for goodness of fit. There is sufficient documentation available on how to do this manually (by hand) so you can write a function to do this by yourself.