Minimize cost and maximize quality of function in four variables
So, simplifying a bit, you want to maximize an objective function $\Phi(w,x,y,z)$, where $\Phi$ has the following properties:


$\Phi(w,x,y,z)$ is monotonically increasing in $w$, in $x$, and in $y$
for any fixed $w,x,y$, $\Phi(w,x,y,z)$ is monotonically increasing-then-decreasing in $z$, i.e., there exists some $z_0$ such that $\Phi(w,x,y,z)$ is monotonically increasing in $z$ for $z \le z_0$ and is monotonically decreasing in $z$ for $z \ge z_0$.


Here $\Phi(w,x,y,z)$ represents the accuracy.

There are efficient algorithms to maximize this objective function, if $\Phi$ has some additional nice properties.  I'll describe how to do so, in several steps.

Given $w,x,y$, it is possible to efficiently find the $z$ that maximizes $\Phi(w,x,y,z)$ using ternary search (see also here).  Let $\Phi_3(w,x,y)$ denote the value of $\Phi(w,x,y,z)$ for the value of $z$ that maximizes  $\Phi(w,x,y,z)$, i.e.,

$$\Phi_3(w,x,y) = \max_z \Phi(w,x,y,z).$$

The comment above indicates that you can efficiently compute $\Phi_3(w,x,y)$ using ternary search.  If $z$ comes from a discrete space with $N$ possible values, the running time to compute $\Phi_3(w,x,y)$ will be $\Theta(N)$; if it comes from a continuous space, you can find the approximate value of $\Phi_3(w,x,y)$, and for typical functions, it the convergence rate will often be very fast (the number of iterations will be proportional to the number of digits of accuracy you want in the answer).

Now, if we are lucky, $\Phi_3$ will be monotonically increasing in $w,x,y$.  If this happens, we can continue to apply a similar idea, as follows.

Under this condition, given $w,x$, it is possible to efficiently find the $y$ that maximizes $\Phi_3(w,x,y)$.  How?  Use binary search on $y$, and use the fact that we know how to efficiently compute $\Phi_3(w,x,y)$.  So, define

$$\Phi_2(w,x) = \max_y \Phi_3(w,x,y),$$

and note that this gives us an efficient way to compute $\Phi_2(w,x)$.  The running time will again be efficient, i.e., $\Theta(\lg N)$ evaluations of $\Phi_3$ if $y$ comes from a discrete space with $N$ possible values, otherwise we can get an approximate optimal value efficiently (typically, the number of evaluations of $\Phi_3$ needed is proportional to the number of digits of accuracy we want in the answer).

Suppose that $\Phi_2(w,x)$ is monotonically increasing in $w,x$ and $\Phi_1(w) = \max_x \Phi_2(w,x)$ is monotonically increasing in $w$.  Then Applying the same techniques, we can see that given $w$ it is possible to efficiently find the $x$ that maximizes $\Phi_2(w,x)$ (using binary search), and thus that it is possible to find the $w$ that maximizes $\Phi_1(w)$. Finally, the maximum value of $\Phi_1(w)$ is exactly the maximum value of $\Phi(w,x,y,z)$ over all $w,x,y,z$.

So if $\Phi$ has some additional nice properties, there are efficient algorithms.  The resulting algorithm will have running time $O(k^4)$, where $k = \lg N$ if each parameter comes from a discrete space; or where $k$ is typically proportional to the number of bits of accuracy desired if the parameter comes from a continuous space.

Therefore I suggest you try this and see if your $\Phi$ has the required nice properties.  If it does, you are in good shape and you can try a technique based on this.



What do you do if $\Phi$ doesn't have these nice properties?  Probably your best next bet is to try some standard black-box mathematical optimization routine, e.g., hill-climbing, gradient descent, or convex optimization.  If $\Phi$ is a relatively smooth function, I suspect they might work pretty well.