PERF: to_datime fastpath for %Y%m%d is slower
We have a check for whether `format == '%Y%m%d'`, but this actually seems to be slower:

```python
In [86]: s = pd.Series(['20120101']*1000000)

In [87]: %timeit pd.to_datetime(s)
229 ms ± 12.5 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

In [88]: %timeit pd.to_datetime(s, format='%Y%m%d')
749 ms ± 67.4 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
```