Do I have to buy all new components if I were to get a RAMPS 1.4 that supports 24 V power?
You need all components that are supplied voltage by the RAMPS board to be able to deal with a 24V input. Some of your parts are most likely compatible, as the stepper drivers. Others get the 5V from the Arduino, as the endstops. Some will most likely need replacement, as


the Hotend heater cartridge. It will need to be exchanged. You can easily buy those in 12V and 24V versions and a 12V version should not get 24V.
the fans. Chances here are about as close to 100% as it gets that they need to be replaced. They are directly connected to the 12V, some are PWM regulated but that doesn't change that they need to be replaced with proper 24V fans.
the heated bed. Since it gets direct power from the board you cannot use your old 12V bed. This is unless you had your old bed switched by a solid state relay (SSR) before. Then you'd have to check the SSR's switching signal tolerances.
anything else you might have hooked up to your power supply directly (lights for example) should be checked for input tolerance.