CLN: remove unnecessary get_value_at calls
I found in another branch that these calls didn't play nicely with `nogil`, so went through and removed the ones that are unnecessary.  Turns out we really only need the one in libindex.get_value_at, since that is the only place where we _arent_ assured that `loc` is intlike.