BUG: Series.__getitem__ with downstream scalars
- [ ] closes #xxxx
- [ ] tests added / passed
- [ ] passes `black pandas`
- [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
- [ ] whatsnew entry

cc @spencerclark I think this fixes a subset of the issues reported in https://github.com/pydata/xarray/issues/3751, can you confirm?

@jorisvandenbossche IIRC geopandas scalars not being recognized by lib.is_scalar has caused some issues there; does this address any of those?