from the population (1,1,3,4,0,2,1,0), then the sample mean is , as compared to the population mean of . Even if a sample is random, it is rarely perfectly representative, and other samples would have other sample means even if the samples were all from the same population. The sample (2, 1, 0), for example, would have a sample mean of 1. If the statistician is interested in K variables rather than one, each observation having a value for each of those K variables, the overall sample mean consists of K sample means for individual variables. Let be the ith independently drawn observation (i=1,...,N) on the jth random variable (j=1,...,K). These observations can be arranged into N column vectors, each with K entries, with the K×1 column vector giving the i-th observations of all variables being denoted (i=1,...,N). The sample mean vector is a column vector whose j-th element is the average value of the N observations of the jth variable: Thus, the sample mean vector contains the average of the observations for each variable, and is written Definition of sample covariance The sample covariance matrix is a K-by-K matrix with entries where is an estimate of the covariance between the th variable and the th variable of the population underlying the data. In terms of the observation vectors, the sample covariance is Alternatively, arranging the observation vectors as the columns of a matrix, so that , which is a matrix of K rows and N columns. Here, the sample covariance matrix can be computed as , where is an N by vector of ones. If the observations are arranged as rows instead of columns, so is now a 1×K row vector and is an N×K matrix whose column j is the vector of N observations on variable j, then applying transposes in the appropriate places yields Like covariance matrices for random vector, sample covariance matrices are positive semi-definite. To prove it, note that for any matrix the matrix is positive semi-definite. Furthermore, a covariance matrix is positive definite if and only if the rank of the vectors is K. Unbiasedness The sample mean and the sample covariance matrix are unbiased estimates of the mean and the covariance matrix of the random vector , a row vector whose jth element (j = 1, ..., K) is one of the random variables. The sample covariance matrix has in the denominator rather than