PERF: Fixed performance regression in Series init
Closes https://github.com/pandas-dev/pandas/issues/30564

PR

```
[ 50.00%] ··· series_methods.SeriesConstructor.time_constructor                                                                                                                   ok
[ 50.00%] ··· ====== ==========
               data
              ------ ----------
               None   670±0μs
               dict   84.4±0ms
              ====== ==========


```

master

```
[ 50.00%] ··· series_methods.SeriesConstructor.time_constructor                                                                                                                   ok
[ 50.00%] ··· ====== =========
               data
              ------ ---------
               None   756±0μs
               dict   725±0ms
              ====== =========
```

No whatsnew, since the regression was only on master.