How to use the finite difference to compute gradient for very complex loss function I need to put the prediction into a complex function to calculate loss.
This means that i can't build the loss function by tensorflow's operator and can't get  gradient from Automatic Differentiation.

I want to get gradient from finite difference in my training step.
You may consider using a score function estimator (also known as REINFORCE), which defines an estimator of the gradient of a scoring function that does not need to be differentiable. This can be achieved thanks to the log derivative trick.

You may find its mathematical foundations in the original REINFORCE article by Williams or in Ian Goodfellow's seminal book on Deep Learning (section 20.9.1 Back-Propagating through Discrete Stochastic Operations).