I want to perform gradient descent optimization of the probability of a sample under a multivariate normal probability density function. For your convenience I state the PDF here: $$\mathcal{N}(\boldsymbol\mu,\,\boldsymbol\Sigma) = (2\pi)^{-\frac{k}{2}}|\boldsymbol\Sigma|^{-\frac{1}{2}}\, e^{ -\frac{1}{2}(\mathbf{x}-\boldsymbol\mu)'\boldsymbol\Sigma^{-1}(\mathbf{x}-\boldsymbol\mu) }$$The probability of a sample under this model for the special case of zero mean and unit covariance is given by$$p(\vec{x}) = \frac{1}{(2\pi)^{-n/2}}\prod_{i=1}^n e^{-\frac{x_i^2}{2}}$$where $n$ is the number of entries in $\vec{x}$. $\vec{x}$ is the projection of a sample onto a PCA model in each iteration and so should actually be seen as a function of a different set of variables $\vec{y}$ that are parametrizing the original problem. In addition, I would like to work with the log of the probability for numerical stability. All in all, I'm looking for the derivative$$\frac{d}{d\vec{y}}\log p(\vec{x(\vec{y}})) = \frac{d}{d\vec{y}}\log \frac{1}{(2\pi)^{-n/2}}\prod_{i=1}^n e^{-\frac{\vec{x_i(\vec{y}})^2}{2}}$$and this where I'm stuck. Help and pointers are really appreciated. So my questions are these:(1) Is this approach to deriving a gradient descent cost function correct?(2) What is the derivative given by?
Gradient descent on the PDF of the multivariate normal distribution
optimization
Start by simplifying your expression by using the fact that the log of a product is the sum of the logarithms of the factors in the product.  The resulting expression is a quadratic form that is easy to differentiate.  