Replacement of a safety capacitor
The fact that you are having to replace a Y class capacitor means that it failed in a benign manner without catching fire or smoke. You can't say the same for X class capacitors because these can fail short circuit.
In addition, you can use a Y class capacitor in front of a fuse but you can't with an X class capacitor hence, what you propose could be very risky. Y class capacitors are designed to fail open-circuit.
A Y2 capacitor is rated to withstand a peak impulse of 5 kV. An X2 capacitor is only rated for 2.5 kV peak impulse.