What could be a dataset in which the presence of an outlier dramatically affects the performance of Ordinary Least Squares (OLS) regression?
If you are looking for a real-world data set here is one on Harvard's Dataverse that examines state social politics research for outliers for the same purpose you are looking for. If you are looking for one for more illustrative purposes one data set worth knowing is Anscombe's quartet for demonstrating how misleading some descriptive statistics can be. For your own investigations, many data sets with and without outliers can be found on Google's beta dataset search and are worth exploring if you are curious!