Several datetime tests fail on arm64, armel, armhf, mips, mips64el, mipsel, s390x. NaT seems to be handled differently.

For example this chunk of unit test code from pandas.tests.frame.test_indexing.py test_where_datetime

```python
from datetime import datetime
from pandas import DataFrame, date_range
import numpy as np
from pandas.util.testing import assert_frame_equal

df = DataFrame(dict(A=date_range('20130102', periods=5),
                    B=date_range('20130104', periods=5),
                    C=np.random.randn(5)))

stamp = datetime(2013, 1, 3)
result = df[df > stamp]
expected = df.copy()
expected.loc[[0, 1], 'A'] = np.nan
assert_frame_equal(result, expected)
```

produces these result & expected on arm64
```
>>> result
           A          B         C
0 1970-01-01 2013-01-04 -0.238902
1 1970-01-01 2013-01-05  0.055246
2 2013-01-04 2013-01-06  0.583666
3 2013-01-05 2013-01-07 -2.401230
4 2013-01-06 2013-01-08  2.580124
>>> expected
           A          B         C
0        NaT 2013-01-04 -0.238902
1        NaT 2013-01-05  0.055246
2 2013-01-04 2013-01-06  0.583666
3 2013-01-05 2013-01-07 -2.401230
4 2013-01-06 2013-01-08  2.580124
>>> 
```
and these results on amd64
```
           A          B         C
0        NaT 2013-01-04 -1.151324
1        NaT 2013-01-05  0.925227
2 2013-01-04 2013-01-06 -2.267937
3 2013-01-05 2013-01-07  2.129802
4 2013-01-06 2013-01-08  0.103666

In [11]: expected
Out[11]: 
           A          B         C
0        NaT 2013-01-04 -1.151324
1        NaT 2013-01-05  0.925227
2 2013-01-04 2013-01-06 -2.267937
3 2013-01-05 2013-01-07  2.129802
4 2013-01-06 2013-01-08  0.103666
```

The full build log showing more arm64 tests failures is available at:
https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=arm64&ver=0.20.3-2&stamp=1506059703&raw=0 and the buildd status page showing all of the failures & successes:
https://buildd.debian.org/status/package.php?p=pandas

Obviously it'd be nice if the tests behaved consistently independent of architecture.

#### Output of ``pd.show_versions()``

pandas.show_versions() on arm64
<details>
 pandas.show_versions() (failure)

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.4.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.0-3-arm64
machine: aarch64
processor: 
byteorder: little
LC_ALL: C
LANG: C
LOCALE: None.None

pandas: 0.20.3
pytest: 3.2.1
pip: None
setuptools: 36.2.7
Cython: 0.26.1
numpy: 1.13.1
scipy: 0.19.1
xarray: None
IPython: None
sphinx: 1.6.4
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.0
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None
>>> 
</details>

From a working amd64 system:
<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.4.final.0
python-bits: 64
OS: Linux
OS-release: 4.12.0-2-amd64
machine: x86_64
processor: 
byteorder: little
LC_ALL: C
LANG: C
LOCALE: None.None

pandas: 0.20.3
pytest: 3.2.1
pip: None
setuptools: 36.2.7
Cython: 0.26.1
numpy: 1.13.1
scipy: None
xarray: 0.9.2
IPython: 5.1.0
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.0
openpyxl: None
xlrd: 1.0.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
pandas_gbq: None
pandas_datareader: None
</details>

dateutil is 2.6.1 in both environments.
The amd64 test environment is not exactly the same as the buildd. but these tests do pass in the buildd.