REF: shift ravel in infer_dtype
Precursor to #24050 as requested from @jreback in review: https://github.com/pandas-dev/pandas/pull/24050/files#r243769223 / https://github.com/pandas-dev/pandas/pull/24050/files#r243769260
