API: to_datetime inconsistent returning of datetime.datetime
Copying from https://github.com/pandas-dev/pandas/issues/14448#issuecomment-255068330 (as the issue is closed now):

```
In [1]: pd.to_datetime('13000101', errors='ignore')
Out[1]: '13000101'

In [2]: pd.to_datetime('13000101', errors='ignore', format='%Y%m%d')
Out[2]: datetime.datetime(1300, 1, 1, 0, 0)
```

The above inconsistency (which is also not documented at all I think), is that something we want to fix?
