Using a 0.1uF 275V ceramic capacitor for 5V LDO?
You most certainly can. A ceramic capacitor will actually give you the pretty close to the advertised capacitance if you only apply a small percentage of its rated voltage. 

A ceramic capacitor's effective capacitance degrades with voltage so if you were to apply 275V to your .1uF capacitor, you would unlikely get much more than .05uF of actual capacitance. In your case you will get very close to .1uF. However, your design will be very non-optimal as the size of the capacitor is probably enormous and you'd be better off getting a smaller one.