we have: Identities As noted above, in general, the results of operations will be transposed when switching between numerator-layout and denominator-layout notation. To help make sense of all the identities below, keep in mind the most important rules: the chain rule, product rule and sum rule. The sum rule applies universally, and the product rule applies in most of the cases below, provided that the order of matrix products is maintained, since matrix products are not commutative. The chain rule applies in some of the cases, but unfortunately does not apply in matrix-by-scalar derivatives or scalar-by-matrix derivatives (in the latter case, mostly involving the trace operator applied to matrices). In the latter case, the product rule can't quite be applied directly, either, but the equivalent can be done with a bit more work using the differential identities. The following identities adopt the following conventions: the scalars, a, b, c, d, and e are constant in respect of, and the scalars, u, and v are functions of one of x, x, or X; the vectors, a, b, c, d, and e are constant in respect of, and the vectors, u, and v are functions of one of x, x, or X; the matrices, A, B, C, D, and E are constant in respect of, and the matrices, U and V are functions of one of x, x, or X. Vector-by-vector identities This is presented first because all of the operations that apply to vector-by-vector differentiation apply directly to vector-by-scalar or scalar-by-vector differentiation simply by reducing the appropriate vector in the numerator or denominator to a scalar. {|class="wikitable" style="text-align: center;" |+ Identities: vector-by-vector ! scope="col" width="150" | Condition ! scope="col" width="10" | Expression ! scope="col" width="100" | Numerator layout, i.e. by y and xT ! scope="col" width="100" | Denominator layout, i.e. by yT and x |- | a is not a function of x || ||colspan=2| |- | || || colspan=2| |- | A is not a function of x || || || |- | A is not a function of x || || || |- | a is not a function of x,u = u(x) || | colspan=2| |- | v = v(x), a is not a function of x|| || || |- |v = v(x), u = u(x) || || || |- | A is not a function of x,u = u(x) || || || |- | u = u(x),