OutOfBoundsDatetime for big-endian np.datetime64 arrays
#### Code Sample, a copy-pastable example if possible

```python
ipdb> value = np.array([np.datetime64(1, 'ms')], dtype="<M8[ms]")
ipdb> pd.Series(value)
0   1970-01-01 00:00:00.001
dtype: datetime64[ns]
ipdb> value = np.array([np.datetime64(1, 'ms')], dtype=">M8[ms]")
ipdb> pd.Series(value)
*** pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 2285384-04-02 23:52:07
```
#### Problem description

Seems like something weird is going on when converting the big-endian timestamp, which does not happen for the little-endian one.

#### Expected Output
The same as for the little-endian data type.

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]

ipdb>  pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.3.final.0
python-bits      : 64
OS               : Darwin
OS-release       : 18.5.0
machine          : x86_64
processor        : i386
byteorder        : little
LC_ALL           : en_US.UTF-8
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 0.25.3
numpy            : 1.17.3
pytz             : 2019.3
dateutil         : 2.8.1
pip              : 19.3.1
setuptools       : 41.6.0.post20191101
Cython           : None
pytest           : 5.0.1
hypothesis       : 4.44.2
sphinx           : 2.2.1
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : 2.10.3
IPython          : 7.9.0
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : 0.13.0
pytables         : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None
</details>
