title: Reduce the size of a large data set by sampling/interpolation to improve chart performance
body: <p>I have a large set (>2000) of time series data that I'd like to display using d3 in the browser.  D3 is working great for displaying a subset of the data (~100 points) to the user, but I also want a "context" view (<a href="http://bl.ocks.org/mbostock/1667367" rel="noreferrer">like this</a>) to show the entire data set and allow users to select as subregion to view in detail.  </p>
  
  <p>However, performance is abysmal when trying to display that many points in d3.  I feel like a good solution would be to select a sample of the data and then use some kind of interpolation (spline, polynomial, etc., this is the part I know how to do) to draw a curve that is reasonably similar to the actual data.</p>
  
  <p>However, it's not clear to me how I ought to go about selecting the subset. The data (shown below) has rather flat regions where fewer samples would be needed for a decent interpolation, and other regions where the absolute derivative is quite high, where more frequent sampling is needed.  </p>
  
  <p>To further complicate matters, the data has gaps (where the sensor generating it was failing or out of range), and I'd like to <em>keep</em> these gaps in the chart rather than interpolating through them. Detection of the gaps is fairly simple though, and simply clipping them out after drawing the entire data set with the interpolation seems like a reasonable solution.</p>
  
  <p>I'm doing this in JavaScript, but a solution in any language or a mathematical answer to the problem would do.  </p>
  
  <p><img src="https://i.stack.imgur.com/FnN3e.png" alt="the data in question"></p>