matplotlib major display issue with dense data sets
Question: I've run into a fairly serious issue with matplotlib and Python. I have a
dense periodogram data set and want to plot it. The issue is that when there
are more data points than can be plotted on a pixel, the package does not pick
the min and max to display. This means a casual look at the plot can lead you
to incorrect conclusions.

Here's an example of such a problem:  
![example](http://i.stack.imgur.com/UEBUI.png)

The dataset was plotted with `plot()` and `scatter()` overlayed. You can see
that in the dense data fields, the blue line that connects the data does not
reach the actual peaks, leading a human viewer to conclude the peak at ~2.4 is
the maximum, when it's really not.

If you zoom-in or force a wide viewing window, it is displayed correctly.
`rasterize` and `aa` keywords have no effect on the issue.

Is there a way to ensure that the min/max points of a `plot()` call are always
rendered? Otherwise, this needs to be addressed in an update to matplotlib.
I've never had a plotting package behave like this, and this is a pretty major
issue.

Edit:

    
    
    x = numpy.linspace(0,1,2000000)
    y = numpy.random.random(x.shape)
    y[1000000]=2
    
    plot(x,y)
    show()
    

Should replicate the problem. Though it may depend on your monitor resolution.
By dragging and resizing the window, you should see the problem. One data
point should stick out a y=2, but that doesn't always display.


Answer: This is due to the path-simplification algorithm in matplotlib. While it's
certainly not desirable in some cases, it's deliberate behavior to speed up
rendering.

The simplification algorithm was changed at some point to avoid skipping
"outlier" points, so newer versions of mpl don't exhibit this exact behavior
(the path is still simplified, though).

If you don't want to simplify paths, then you can disable it in the rc
parameters (either in your `.matplotlibrc` file or at runtime).

E.g.

    
    
    import matplotlib as mpl
    mpl.rcParams['path.simplify'] = False
    import matplotlib.pyplot as plt
    

However, it may make more sense to use an "envelope" style plot. As a quick
example:

    
    
    import matplotlib.pyplot as plt
    import numpy as np
    
    def main():
        num = 10000
        x = np.linspace(0, 10, num)
        y = np.cos(x) + 5 * np.random.random(num)
    
        fig, (ax1, ax2) = plt.subplots(nrows=2)
        ax1.plot(x, y)
        envelope_plot(x, y, winsize=40, ax=ax2)
        plt.show()
    
    def envelope_plot(x, y, winsize, ax=None, fill='gray', color='blue'):
        if ax is None:
            ax = plt.gca()
        # Coarsely chunk the data, discarding the last window if it's not evenly
        # divisible. (Fast and memory-efficient)
        numwin = x.size // winsize
        ywin = y[:winsize * numwin].reshape(-1, winsize)
        xwin = x[:winsize * numwin].reshape(-1, winsize)
        # Find the min, max, and mean within each window 
        ymin = ywin.min(axis=1)
        ymax = ywin.max(axis=1)
        ymean = ywin.mean(axis=1)
        xmean = xwin.mean(axis=1)
    
        fill_artist = ax.fill_between(xmean, ymin, ymax, color=fill, 
                                      edgecolor='none', alpha=0.5)
        line, = ax.plot(xmean, ymean, color=color, linestyle='-')
        return fill_artist, line
    
    if __name__ == '__main__':
        main()
    

![enter image description here](http://i.stack.imgur.com/xBZJ5.png)

