Smart Light Switch power supply solution for 2-wire (without neutral) scenarios
What kind of smart switch design can bypass this issue?


I have not seen a Wi-Fi switch that doesn’t need neutral. Perhaps this can be done, but I guess the power consumption is such that manufacturers choose different protocols.
There are popular generic RF (433 mhz, etc.), z-wave and ZigBee switches that work without neutral. All those protocols are designed around low-energy standards. Some can be used with a generic X to Wi-Fi gateway which is powered from mains.


  Why do some bulbs require a capacitor in parallel for non-neutral-wire switches to work?


The switch requires some current to pass even in “off” state. If the bulb doesn’t let the current pass or if it starts to flicker as a result of this current, then you will be instructed to install a capacitor in parallel to the bulb.


  Why does it solve the problem?


The capacitor forms a capacitive dropper (together with some components in the switch) that bypasses the bulb. A capacitor in an AC circuit forms a current limiter because when current flows one way the capacitor lets it pass until the capacitor is charged in one polarity and then when the current is reversed the capacitor discharged and lets the current flow the other way. The amount of current depends on the capacity of the capacitor and the frequency of the AC.

Also see this relevant discussion.


  Why isn’t this standard in all bulbs?


Because it would waste energy. Unless you need some current to pass constantly to power this special kind of switch you shouldn’t install such a capacitor with the bulb.