Walls on cross section of a solidified object look hollow
The only way to do this is by using a boolean modifier (which is non-destructive and reversible, until you apply the modifier, so just don't ever apply the modifier).  The object that you use to drive the boolean can be hidden the whole time, and you can quickly toggle the visibility of the effects of the boolean modifier by clicking the buttons at the top of the modifier; camera button disables visibility in the render, and desktop button disables visibility in viewport.  So you can just always keep the render visibility off, and keep the viewport visibility usually off but quickly toggle it on for those screenshots by just clicking the button and then toggle it back off.  It's not as quick as Alt-B, but it effectively does exactly what you're explaining you want in the question and in your comments.