How do I get the menu block choice "Expand all menu items" to stay checked
That is not the default behavior. I suppose you have installed Menu Block module.

In that case the "expand all" option from Drupal Core does not seem to work, you will need to use the "expand all" option from the Menu Block module. You can find this at "Advanced options" in the block configuration.