Regression on extreme values I have a data set that looks like this:



I would like to estimate a relationship between x-values and the corresponding 5% extreme y-values, something that might look like that :



Do you have an idea of an algorithm that might help me for this ? I thought about labelling the extreme values for later finding a separating hyperplane, but I have no clue on how to label these "extreme values" (I cannot just take the 5% lowest and highest values as all these would end up in the same region).

Thanks for your ideas !
I don't know of a pre-canned algorithm but I would just sweep on the angle from zero to ninety degrees with a triangular region and count the points.  For each step in the sweep, record the angle and the count.  When the sweep is done you will have an array of angles with bin counts and then you can convert to percentage of total count.  You will have to figure out a resolution of the triangle's angle so that it is fine enough to capture less than 5% of the points.