Best-practices for potting a high-voltage circuit
Increase the diameter of the conductors. This will decrease the electric field strength at the surface. It sounds like you only need to halve the field strength, so only need to double the radius of curvature.
Where you use connecting, use thicker wire. Where that wire is a component lead, then you could wrap some wire or foil around it, or slip a metal tube over it. If you have soldered joints, make sure you have not left any sharp edges or points where you've cut wires. Use some extra solder and let it ball up under its own surface tension.