BUG: Setting timedelta column by index causes it to loss precision
#### Code Sample, a copy-pastable example if possible

```python
dt1 = pd.Timedelta(0)
dt2 = pd.Timedelta(28767471428571405)
df = pd.DataFrame({'dt': pd.Series([dt1, dt2])})
df_copy = df.copy()
s = pd.Series([dt1])

expected = df['dt'].iloc[1].value
df.loc[[True, False]] = s
result = df['dt'].iloc[1].value

print(result)
#28767471428571404
```
#### Problem description

Setting a timedelta column by index causes it to be casted to double and back. Therefore, it will loss precision.

#### Expected Output
28767471428571405

#### Output of ``pd.show_versions()``
0.23.4

<details>

[paste the output of ``pd.show_versions()`` here below this line]

</details>
