Using the trainbr function for classification in Matlab
The trainbr mode uses the Bayesian regularization backpropagation. This method was presented in 1, which presents a regression problem with the loss function
$$ E_D = \sum_{i=1}^n (t_i - a_i)^2 $$
where $t_i$ is the target and $a_i$ is the network's response. The paper proposes to add a regularization term, leading to a loss function $F$ of the form
$$ F = \beta E_D + \alpha E_W $$
where $E_W$ is the square of the sums of all network weights, i.e. $E_W = \sum_{i,j} \| w_{ij} \|^2$.
The two parameters $\alpha$ and $\beta$ control the weighing of the two parts $E_D$ and $E_W$: For $\alpha \ll \beta$, the network will minimize the loss, without really trying to keep weights low. For $\alpha \gg \beta$, the network will minimize the weights, allowing for some more error. In reality, this means a large $\alpha$ will stop the network from overfitting, which leads to a better generalization at the cost of a larger training error.

The key to find a train a model which generalizes well, but still has a low error rate, is the right setting of $\alpha$ and $\beta$. This is achieved by treating them as random variables and finding an optimal setting, using the Bayesian methods presented in 2. (I won't talk about the details on that here, you can find that in the two linked papers.) Finally, the paper presents an algorithm, which calculates the optimal $\alpha$ and $\beta$ in each training iteration. This makes this algorithm generalize really well, especially in the presence of noisy input signals.

However, as described, the loss function is a weighted sum between the MSE ($E_D$) and the regularization term ($E_W$). So, in short: you can only use it with the MSE, and not with cross-entropy. You'll need a different training algorithm, you can find a list in the MATLAB documentation, here.

References:


  1 F. D. Foresee and M. T. Hagan: "Gauss-Newton Approximation to Bayesian Learning", in Proceedings of the 1997 International Joint Conference on Neural Networks, June 1997. DOI: 10.1109/ICNN.1997.614194. [Link to PDF].
  
  2 D. J. C. McKay: "Bayesian Interpolation", Neural Computation, May 1992, Vol. 4, No. 3, pp. 415-447. DOI: 10.1162/neco.1992.4.3.415. [Link to PDF].