BUG: pd.to_datetime() throws if caching is on with Null-like arguments
- [x] closes #22305
- [x] tests added / passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [x] whatsnew entry

The problem was in contradictory behavior between `pd.Index([pd.NaT, None]).is_unique` that returns `False` and `algorithms.unique([pd.NaT, None])` that returns `array([NaT, None], dtype=object)` equivalent to the object being unique.

We are only using `Index(arg).unique()` now.

The test written by @realead was added to verify changes.