How would I go about reverse-engineering RF lights?
start with a teardown, either do your own, or view one by an "expert" on you-tube.

you need to know three things,  what the RF frequency is, what the modulation scheme is and what the data protocol is

It may be that the makers have used a common RF chip like NRF2201 in which case you could start with a NRF2201 module and need less detailed knowledge of the RF protocol...