Revert "REF: back IntervalArray by a single ndarray (#37047)"
This reverts commit 9cb372376fc78aa66e2559de919592007b74cfaa.

See discussion in https://github.com/pandas-dev/pandas/pull/37047/

@jbrockmendel can you give this a check? (there were already several conflicts)