macOS: Terminal scrolls past file boundaries
I managed to eventually find the settings to stop this behavior.

In Terminal, it was Preferences -> Profiles -> Keyboard and check the box for "scroll alternate screen"

In iTerm2, it was Preferences -> Advanced -> set "Scroll wheel sends arrow keys when in alternate screen mode" to Yes.