Why did Shimano eliminate the B tension spring on the latest rear derailleurs?
There are no recent Shimano rear derailers without b-tension (angle) adjustment, other than maybe very bottom-end ones, but there are many where the look and location of it is different from older (namely pre-Shadow) models. RD-M8000 absolutely has a b-tension screw, although in the exploded view document they call it an "End Adjust Bolt" for no good reason.



Here is my understanding of the design goals and tradeoffs involved with sprung upper pivots:

The one and only reason you would want a sprung upper pivot is to maintain chain wrap and pulley/cog distance when shifting in front on a derailer where the guide pulley and cage pivot are offset (as opposed to coaxial as on most current MTB derailers, and including M8000). If you can make the cage pivot and the guide pulley coaxial, there is no movement of the pulley as the cage moves from shifting the front derailer. The reason for wanting the offset also has to do with getting sufficient chain wrap given the space you're working with and the gears you want access to. Were it not for this, all the spring would do is add chain slap, resistance, weight, and complexity.

The Shadow design, the larger pulley sizes now used, and the new emphasis on single or double as opposed to triple ring setups all create a dynamic where Shimano has decided they're better off without sprung upper pivots on MTB RDs. SRAM has done the same. There may also be an aspect to it where the sprung pivot plus offset guide pulley design gets impractial with very wide range clusters - I'm not sure of that but I believe it starts requiring a lot more offset to work. 

It's nothing new. Despite being out of favor for most of the past 4 or so decades, there have been a lot of unsprung upper pivot rear derailers. The spring's necessity is something manufacturers have been historically on the fence about.