REF: simplify _cython_agg_blocks
Orthogonal to #35839, though a rebase will be needed.

cc @TomAugspurger did you already try this in #31616?  If so, we need to identify a test case in which this doesnt work