likelihoods different methods such as Laplace approximation and variational methods are needed to approximate the estimators. Example kernels Separable A simple, but broadly applicable, class of multi-output kernels can be separated into the product of a kernel on the input-space and a kernel representing the correlations among the outputs: : scalar kernel on : scalar kernel on In matrix form: where is a symmetric and positive semi-definite matrix. Note, setting to the identity matrix treats the outputs as unrelated and is equivalent to solving the scalar-output problems separately. For a slightly more general form, adding several of these kernels yields sum of separable kernels (SoS kernels). From regularization literatureC.A. Micchelli and M. Pontil. On learning vector–valued functions. Neural Computation, 17:177–204, 2005.T.Evgeniou, C.A.Micchelli, and M.Pontil. Learning multiple tasks with kernel methods. Journal of Machine Learning Research, 6:615–637, 2005. Derived from regularizer One way of obtaining is to specify a regularizer which limits the complexity of in a desirable way, and then derive the corresponding kernel. For certain regularizers, this kernel will turn out to be separable. Mixed-effect regularizer where: where matrix with all entries equal to 1. This regularizer is a combination of limiting the complexity of each component of the estimator () and forcing each component of the estimator to be close to the mean of all the components. Setting treats all the components as independent and is the same as solving the scalar problems separately. Setting assumes all the components are explained by the same function. Cluster-based regularizer where: is the index set of components that belong to cluster is the cardinality of cluster if and both belong to cluster ( otherwise where This regularizer divides the components into clusters and forces the components in each cluster to be similar. Graph regularizer where matrix of weights encoding the similarities between the components where , Note, is the graph laplacian. See also: graph kernel. Learned from data Several approaches to learning from data have been proposed. These include: performing a preliminary inference step to estimate from the training data, a proposal to learn and together based on the cluster regularizer, and sparsity-based approaches which assume only a few of the features are needed. From Bayesian literature Linear model of coregionalization (LMC) In LMC, outputs are expressed as linear combinations of independent random functions such that the resulting covariance function (over all inputs and outputs) is a valid positive semidefinite function.