How to use the finite difference to compute gradient for very complex loss function
You may consider using a score function estimator (also known as REINFORCE), which defines an estimator of the gradient of a scoring function that does not need to be differentiable. This can be achieved thanks to the log derivative trick.

You may find its mathematical foundations in the original REINFORCE article by Williams or in Ian Goodfellow's seminal book on Deep Learning (section 20.9.1 Back-Propagating through Discrete Stochastic Operations).