What kind of optimization problem this belongs to?
Yes, many standard methods can handle this.  Your score is called a loss function.  In particular, let $\ell(x_i,y_i)$ denote the penalty (score) if on input $x_i$, your decision rule outputs $y_i$.  If $x_1,\dots,x_n$ is a training set, we define the total loss over the training set (the empirical risk) of a decision rule $f$ to be

$$L(f) = \sum_{i=1}^n \ell(x_i,f(x_i)).$$

Now your goal is to find $f$ that minimizes the total loss $L(f)$.  Many machine learning algorithms can be used in this manner, with an arbitrary loss function.  Typically, we require that the loss function be differentiable, and then we use gradient descent to find $f$ that minimizes $L(f)$.  For instance, if you are using neural networks, then you use gradient descent to find weights for the neural network that minimize $L(f)$.  Often you add a regularization term to the loss function, too, to prevent overfitting.