pd.to_timedelta unit ignored for string input
```python
In : pd.to_timedelta(123456789, unit='s')
Out: Timedelta('1428 days 21:33:09') # ok

In : pd.to_timedelta('123456789', unit='s')
Out: Timedelta('0 days 00:00:00.123456') # oops!

In : pd.to_timedelta('123.45', unit='s')
Out: ValueError('no units specified') # ???
```
#### Problem description

Evidently, the `unit` parameter is ignored when the input is a string (yet strings that look like numbers are accepted as if they were actually numbers).

Reproduced in Pandas 0.21.1 and 0.23.4.