BUG/TST: Fix infer_dtype for Period array-likes and general ExtensionArrays
Closes #23553

In addition, I also changed to let infer_dtype fall back to inferring from the scalar elements if the array-like is not recognized directly (now it raises an error, which doesn't seem very useful?)