Proof of work - lottery approach
There's a lot going, but I'll do my best.


  QUESTION: Can the bitcoin network randomly select a subset of the existing nodes, for each block, who then compete to solve the hash?


Selecting a subset of existing nodes isn't easy as it seems. The vast majority of bitcoin nodes aren't reachable at all, in fact. That's why every website that counts those are different from each other. (The best probably being Luke Dashjr's one).

So you might: "Well, you could make it so that in order to participate you have to public announce your node". But that opens up (from the top of my head) a big attack vector: DDoS's. If every node participating in the consensus is known and reachable then you could hit them once they are selected and cause a network failure.

But that's not even the most important part: This approach would be extremely easy to gamble. Any person could run thousands or millions of nodes (Amazon Web Service has 1 million business customers) and change the odds to their favor.

Proof-of-Work is not a consensus algorithm, but rather an Anti-Sybil protocol. Sybil being the attack that I described in the last paragraph.


  Can the lottery fee approach achieve the same goal without sacrificing
  security?


Absolutely and 100% no, if you take "without sacrificing security" serious. And (1) I'm not sure if it can be done at all or (2) if it would work, but regardless: no. 


  The result is a huge energy burn


This is a little off-topic but I feel that might be worth commenting. Bitcoin energy usage is far from "deadly" or something like that. Most researches point to +70% use of renewable energy in the industry, highest than any other.