Which Optimization method to use?
Bayesian optimization is a principled way of sequentially finding the extremum of black-box functions. What's more, there a numerous software packages that make it easy, such as BayesOpt and MOE. Another flexible Bayesian framework that you can use for optimization is Gaussian processes: Global Optimisation with Gaussian Processes