How to split up a View up amongst different Panels panes? Context display?
The module I needed to enable was Views content panes. I was looking for something like it in the Views and Panels section of the modules list - it was in neither, it's bundled with ctools and listed under Chaos Tools Suite.

Then, once enabled, you get two new Views display mode options:


Context is the one to allow a view to be loaded as a context behind a panel.
Content pane is like a smart block where you can allow settings to be set per panel pane (e.g. offset and pager, allowing the view to be sliced up), toggling fields on and off, changing titles, etc. A few gotchas: 


All such options are disabled from being edited in Panels by default, you have to enable them each time using the "Allow settings" option in the middle in Views
Panels can be a bit fussy about showing newly added view panes. You might need to save/update the panel/page manager page before it'll register the new view displays, even though it seems to load everything new each time. If even that doesn't work, try clearing caches.