PERF: regression in time series plotting
Opening a dedicated issue to not have this lost in https://github.com/pandas-dev/pandas/issues/18532.

Using the example from the benchmarks:

```
N = 2000 
M = 5 
idx = date_range('1/1/1975', periods=N) 
df = DataFrame(np.random.randn(N, M), index=idx)  
```

I get on master:
```
In [25]: %%timeit plt.close('all') 
    ...: df.plot() 
                          
2.07 s ± 251 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```

but on 0.23.4:

```
In [2]: %%timeit plt.close('all') 
   ...: df.plot()
   ...: 
87.4 ms ± 979 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
```

The slowdown was originally even bigger and was addressed already partly https://github.com/pandas-dev/pandas/pull/23589, but there is still a \~ 50x slowdown remaining.
