to_datetime errors='ignore' unexpected behavior
I'm trying to apply pd.to_datetime on an integer column to convert integers (including missing indicators) to datetimes. In order to preserve all the missing indicators, ideally, I would prefer setting the "errors" parameter to "ignore" to keep all the invalid parsings. But then I noticed that setting errors='ignore' not only preserved the missing indicators, but kept everything the same, even the dtype of the input column. See below for more details.  
 
```python
print(dt_col)
```
<Outputs>
0    20181223

1    20161017
2      -99998
3    20171220
4    20170511
Name: dt_col, dtype: int64
</Outputs>

```python
pd.to_datetime(dt_col, format='%Y%m%d', errors='ignore')
```
<Outputs>
0    20181223

1    20161017
2      -99998
3    20171220
4    20170511
Name: dt_col, dtype: int64

</Outputs>

```python
pd.to_datetime(dt_col, format='%Y%m%d', errors='coerce')
```
<Outputs>
0   2018-12-23

1   2016-10-17
2          NaT
3   2017-12-20
4   2017-05-11
Name: dt_col, dtype: datetime64[ns]

</Outputs>


Pandas version: 
- pandas==0.25.1

Is anyone else seeing the same behavior? 