PERF: Index.__getitem__ should not need to do dtype inference
Currently, the `__getitem__` call in a snippet like

```
idx = pd.Index([1, 2, 3, 4])
idx[np.array([0, 3])]
```

does type inference when creating the new Index object (using `_shallow_copy_with_infer`), while I think we can always know for sure the result will be another Int64Index (for the above example) ?