Closing a circuit a predefined number of times
First, you don't need an "Arduino", its just like a development platform. You can use just an micro-controller, like the one used in some Arduino, ATMega328P, or one of lower cost, just a bit more or not, than you need for a discrete solution.

Anyway, yes that's overkill for using a micro-controller, you can use a CD4017 counter or other family, and a oscillator, can be with a 555 or other IC, or you can do it with discrete components, like transistors, and just use one of the CD4017 outputs to stop the oscillator, you can then use the oscillator pulses to close the contacts on the controller.

But that can be difficult, its not always easy to interface to the other circuit.

Other question, is that you need a low voltage DC (near 5V, depending on your circuit). 


Can you get it from the controller? Does the controller will support the additional current demand? 
If not, for low cost you can do a non-isolated one, but I can presume
that in the ambient this is used, it will present some danger, so you
need a isolated power supply.


Like the other answer, if you have engouth knowledge of the circuit in question (that don't need to be a micro-controller), you can try to put a battery on it to preserve its state, a more elegant solution if you know the circuit of the controller.