How to control over-voltage at high voltage?
This is not the complete answer to the question. I will try to clarify the role of ESR in the circuit, since there is a misconception about it in the wording of the question.

Assume zero ESR (i.e. an ideal capacitor) and try to model the circuit like this:







simulate this circuit &ndash; Schematic created using CircuitLab

\$R_{zener}\$ is not an "external" resistor. It is intrinsic series resistance of a non-ideal Zener diode (so called differential resistance). Any real Zener (TVS) diode has non-zero differential resistance, which can be modelled as ideal Zener in series with a resistor. Differential resistance is not a constant; it heavily depends on an operating point (= current through a diode).

What is the maximum voltage the capacitor will be charged to? To answer this question, assume \$V_{spike} &gt; V_{clamp}\$, where \$V_{clamp}\$ is Zener diode clamp voltage.

Then in the steady state the current through the Zener diode will be
$$I_{zener} = \frac{V_{spike} - V_{zener}}{R_{spike} + R_{zener}}$$
by the Ohm's law.
This is the steady state condition, which means the capacitor has been charged to maximum voltage for the given \$V_{spike}\$ level, and no current flows through the capacitor.

The voltage across the capacitor will be
$$V_{cap} = V_{zener} + R_{zener}I_{zener}$$
subsituting \$I_{zener}\$ we get
$$V_{cap} = V_{zener} + \underbrace{(V_{spike} - V_{zener})}_{\text{overvoltage}}\frac{R_{zener}}{R_{spike} + R_{zener}}$$

Your can see that what does matter is the expression
$$\frac{R_{zener}}{R_{spike} + R_{zener}}$$
which is the expression for voltage divider.

If we assume non-zero ESR, it will not affect the formula, since there is no current through the fully charged capacitor, which means no voltage drop across the ESR.

As I have already mentioned in the comment, the ESR does play role if gas discharge tube (GDT) will be used as a protection device. That's because I-V characteristic of a GDT dramatically differs from I-V characteristic of a Zener diode. As soon a GDS breakdown voltage is reached, the discharge begins and the voltage across the tube drops down to tens of volts (arc voltage). Take a look at https://www.bourns.com/pdfs/bourns_gdt_white_paper.pdf . Thus, ESR will limit the GDT current.