Trip RCD with Circuit
Shunt trip breaker

This is the easiest (and most official) way to do this.  A shunt trip breaker sits in your panel (or on your DIN rail) and hooks up the normal way.  It also has two "low voltage" wires on a pigtail.  Apply 24 volts to these for 50ms and the breaker will trip.  

The breaker already has a (very thick, few or even 1 turn) solenoid which operates the magnetic trip mode (intended for shock overloads such as a dead short).  The shunt is adding another coil to that same solenoid, which causes the breaker to trip in magnetic mode. 

You would need to find a shunt-trip breaker that also supports RCD, or just use a separate RCD device. 

There are also remote control breakers which allow a shot of 24V to either trip or reset it. However your chance of finding that in an RCD device is probably ... remote. 

Or... do it properly

You can't use random electronic components (e.g. RU listing) in mains wiring.  You must use proper equipment/devices (e.g. with a UL listing), and you must use it according to instructions.  

One of the doctrines of mains power is that you must keep mains and low voltage separated.  Either the mains power goes inside the junction box and the low-voltage outside, or with a divider within the junction box.

So you need a relay that is listed as equipment (e.g. UL) and ideally has the low voltage and mains wires separate.  This will likely have a 24V coil.  Some relays even contain their own 24VAC transformer with just enough power rating to run the coil - simply shunt two wires and that operates it.  

For instance, Aube makes relay-transformers that mount in a junction box knockout, providing mains inside the box and the low voltage terminals outside the box. Others have a relay &amp; transformer mounted on a junction box cover, again with mains on one side and LV on the other. 

As far as a resistor to induce the ground fault, again the trick is finding one listed as equipment.  Fortunately, the intersection of dimmers and LEDs has created a commercial market for "dummy load" equipment -- such as Lutron's LUT-MLC - and they are implemented as capacitors.  

And as far as inducing a ground fault, do not wire between load (protected) hot and ground.  Ground isn't for that.  Wire between load (protected) hot and line neutral.