Series.__getitem__ materializes Categorical to ndarray
For 

```python
In [12]: c = pd.Series(pd.Categorical(['a'] * 1000))

In [13]: c[0]
```

we hit https://github.com/pandas-dev/pandas/blob/f0cd23ca6485940e3a6572cc4f5afdce4c3921ab/pandas/core/indexes/base.py#L2525-L2543

`_values_from_object` calls `series.get_values()`, which hits `Categorical.get_values`, which coerces to the ndarray of values.

I have a branch based on my ExtensionArray stuff that "fixes" this by seeing if `s` is an instance of `ExtensionArray`, which has the correct semantics for what we need here. But that's not necessarily the best fix here.

master:

```python
In [3]: c = pd.Series(pd.Categorical(['a'] * 1000))

In [4]: %timeit c[0]
50.1 µs ± 1.86 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)
```

My branch:

```python
In [4]: %timeit c[0]
5.76 µs ± 126 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
```