How can I stop brew prune and brew upgrade from fighting?
There is an inelegant solution to this bug/inconsistency in the two functions of the tools: put any file in the directories in question.

Then  brew prune will stop removing them because they will no longer be empty.  If brew prune is no longer removing them, brew upgrade won't create them.  If you're concerned about space, you could even put an alias file in the directories.

In the grand scheme of things, having these empty folders create and then removed is a harmless side effect and doesn’t really break anything functionally.