How necessary is ESD protection, really?
I counted at least five questions in your question. I will try to answer only few.

To begin, there are several levels of ESD events that can be specified by OEM of equipment, for different environment and other operating conditions, all classified in IEC 61000-4-2 standard.

Then yes, connector design does play a significant role in failure rate of electronic equipment. If a connector has a properly routed shield, and signal pins are recessed inside, there is much less chance that the signals will be exposed to direct ESD event, so they might require less level of ESD protection.

Second, TVS diodes do help even if they have clipping voltage of 20-25 V. This is still quite less than the 4 kV discharge from a normal human-body event, so it is quite easier to handle by internal protection.

And yes, in the 80s the feature size of silicon elements of transistors was 2000 nm, today it is much smaller, 1/100 of that, which makes them way more vulnerable to the same ESD energy. And no, there are no modern "5V" MCU, modern MCU are "1V" MCUs. The "5-V tolerant MCU are the blast from the past. There might be "5V" tolerant MCU, but either their functionality is not up to modern IoT demands, or you need to pay a premium for them. 

The rest of questions are insignificant details.

In short, you likely want your product to survive in consumer or industrial environment and don't want to deal with product replacement and associated cost and risking to go out of business. You need to decide, is your business necessary for you? If yes, you don't ask questions and better use all accumulated engineering wisdom to protect your design from ESD.