Timestamp unit is not respected
Timestamps always get returned in nanoseconds regardless of the unit I specify:
```python
import numpy as np, pandas as pd
np.datetime64("2018-01-01").astype("datetime64[s]").astype(int)
>>> 1514764800
pd.Series(np.datetime64("2018-01-01")).astype("datetime64[s]")
```
```
0   2018-01-01
dtype: datetime64[ns]
```
```python
pd.Series(np.datetime64("2018-01-01")).astype("datetime64[s]").astype(int)
```
```
0    1514764800000000000
dtype: int64
```