REF: remove block access in groupby libreduction Series(Bin)Grouper
xref https://github.com/pandas-dev/pandas/issues/39146

The `Series(Bin)Grouper` still has some direct block access, which needs to be resolved to let it work for both Block and ArrayManager