Relay with built in manual switch
The manual lever, which serves as a test button to carry out checks during installation, is not intended for normal use. It will neither set nor reset the relay.
I suppose a toggle switch or an MCB would be a better choice for your application.

But if I use the SPDT momentary toggle switch to set or reset then I would not be able to know which state it was triggered last. Do you have any simple circuit suggestion for this? Like any kind of indicator which says it was in set or reset position?

An indicator lamp 'H1' in the load circuit would do.