Error in pd.to_datetime, with argument np.datetime64 array of 51 elements
#### Code Sample, a copy-pastable example if possible

```python
import pandas as pd
import numpy as np
d = np.datetime64("2000-01-01", "s")

pd.to_datetime(np.full(50, d))
# Works

pd.to_datetime(np.full(51,  d))
# ERROR
# pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 29999249045-07-29 00:00:00



pd.to_datetime(pd.Series(np.full(51,  d)))

```
#### Problem description

With pandas 1.0, we get a new issue when using pd.to_datetime to convert an existing np.datetime64 array with 51 elements. 

This works with 50 elements, or when converting to a Series beforehand, so I believe the problems comes from inside implicit conversion.
