Is it possible to decrease the size of the “activation area” for a hidden Dock in Big Sur?
This is not a real answer.  I am also frustrated by that behaviour but it seems like the scroll bar on some applications can and to avoid BigSur, has to be interacted with above where the actual scroll bar appears to be (ref grg's comment):

Another none answer is to decrease the dock size to the smallest size so that most of the time it's not in the way:

** edit **
I've now moved to dock to the left and made the icons bigger again.