How to restore private file structure after major version update?
Edit: We have a workaround, I hope this question can serve as a good practice guide of good to debug complex software like Drupal showing unexplainable errors or WSOD. Wish we had followed our own advice to not waste time on our futile attempts time.

tl;dr It was the core module Database logging 7.42    Logs and records system events to the database. which is interfering with private files in our specific setting. We disabled most modules including core, and brought them back one-by one. Database logging is reproducibly inducing the error on activation.


Don't panic and make a debugging plan, listing options of what could be checked, and then stick to it until everything has been checked. If out of options, try to shake things up
Compare your system to a vanilla system and note down what is does differently
Modules might interfere with each other in unexpected ways or with your specific configuration. That doesn't mean there is a bug in any of those modules. Boil down your system to the least complex configuration possible, remove any unneeded entry in your configuration file, disable any module that can be disabled including core modules, suspect everyone. 
Check if error still present, if not re-enable all modules one-by-one, and note down which configuration change causes the error. Otherwise, start debugging going to 1.


Cheers