probability that observation i has outcome k, of the following form: where is a regression coefficient associated with the mth explanatory variable and the kth outcome. As explained in the logistic regression article, the regression coefficients and explanatory variables are normally grouped into vectors of size M+1, so that the predictor function can be written more compactly: where is the set of regression coefficients associated with outcome k, and (a row vector) is the set of explanatory variables associated with observation i. As a set of independent binary regressions To arrive at the multinomial logit model, one can imagine, for K possible outcomes, running K-1 independent binary logistic regression models, in which one outcome is chosen as a "pivot" and then the other K-1 outcomes are separately regressed against the pivot outcome. If outcome K (the last outcome) is chosen as the pivot, the K-1 regression equations are: . This formulation is also known as the alr transform commonly used in compositional data analysis. If we exponentiate both sides and solve for the probabilities, we get: Using the fact that all K of the probabilities must sum to one, we find: . We can use this to find the other probabilities: . The fact that we run multiple regressions reveals why the model relies on the assumption of independence of irrelevant alternatives described above. Estimating the coefficients The unknown parameters in each vector βk are typically jointly estimated by maximum a posteriori (MAP) estimation, which is an extension of maximum likelihood using regularization of the weights to prevent pathological solutions (usually a squared regularizing function, which is equivalent to placing a zero-mean Gaussian prior distribution on the weights, but other distributions are also possible). The solution is typically found using an iterative procedure such as generalized iterative scaling, iteratively reweighted least squares (IRLS), by means of gradient-based optimization algorithms such as L-BFGS, or by specialized coordinate descent algorithms. As a log-linear model The formulation of binary logistic regression as a log-linear model can be directly extended to multi-way regression. That is, we model the logarithm of the probability of seeing a given output using the linear predictor as well as an additional normalization factor, the logarithm of the partition function: . As in the binary case, we need an extra term to ensure that the whole set of probabilities forms a probability distribution, i.e. so that they all sum to