the concept of independence can also be formulated for random variables random variables are said to be independent if the events {xi xi }{xin xin are independent for all finite choices of distinct indices in and values xi xin an important characterization of independent random variables is the following (for proofsee [ ]for exampleindependent random variables theorem independence characterization random variables xn with marginal pdfs fx fxn and joint pdf are independent if and only if ( xn fx ( fxn (xn for all xn ( many probabilistic models involve random variables that are independent and identically distributedabbreviated as iid we use this abbreviation throughout this book iid expectation and covariance similar to the univariate casethe expected value of real-valued function of random vector is weighted average of all values that (xcan take specificallyin the continuous caseeh(xh(xf (xdx in the discrete case replace this multidimensional integral with sum using this resultit is not difficult to show that for any collection of dependent or independent random variables xn [ bn xn ex bn exn ( for all constants ab bn moreoverfor independent random variablese[ xn ex ex exn ( we leave the proofs as an exercise the covariance of two random variables and with expectations ux and uy respectivelyis defined as cov(xye[( ux )( uy )this is measure of the amount of linear dependency between the variables let var and var scaled version of the covariance is given by the correlation coefficientcov(xy%(xys sy the following properties follow directly from the definitions of variance and covariance var ex var[ax ba cov(xye[xyux uy cov(xycov(yxcovariance correlation coefficient