ropies, and the joint entropy. Ui=p(xi) vj==(p)yj) ;logvi x1(good) 0.160 0.423 y1(up) 0.300 0.521 x2(same) 0.750 0.311 y2(steady) 0.500 0.500 x3(bad) 0.090 0.313 y3(down) 0.200 0.464 H(X) = 1.0469 H(Y) = 1.4855 H(X)+H(Y) = 2.5324 log ui 0.075 0.280 0.075 0.280 0.010 0.066 0.210 0.473 0.400 0.529 0.140 0.397 0.015 0.091 0.025 0.133 0.050 0.216 0.844 0.942 0.680 H(X, 2.4657 Vji = p(y)|xi) 0.469 0.075 0.082 0.469 0.075 0.082 0.063 0.010 0.040 0.280 0.210 0.386 0.533 0.400 0.363 0.187 0.140 0 0.339 0.167 0.015 0.039 0.278 0.025 0.046 0.556 0.050 0.042 0.506 0.491 = 0.421 H(Y|X) = 1.4188 Wij = p(xilyj) -11i3 log w3j 0.250 0.075 0.150 0.700 0.210 0.108 0.050 0.015 0.065 0.150 0.075 0.205 0.800 0.400 0.129 0.050 0.025 0.108 0.050 0.010 0.043 0.700 0.140 0.072 0.250 0.050 0.100 0.398 0.309 = 0.421 H(X|Y) = 0.9802 H(X,Y) = H(Y|X) + H(X) 2.4657 = 1.4188 + 1.0469 = H(X)Y) + H(Y) = 0.9802 + 1.4855. These two results mean that the prior knowledge of the company's stocks contains an average of 0.0667 bit/symbol of information on the company's quarterly result, and the reverse is also true. As we shall see in the next section, the two differences above are always equal and they are called mutual information. Simply put, the mutual information is the average information that two sources share in common. 5.2 Mutual information Mutual information I introduce yet another type of entropy definition, which will bring us to some closing point and our final reward. We can define the mutual information of two sources X and Y as the bit/symbol quantity: (5.12) We may note the absence of a minus sign in the above definition, unlike in H(X,) Y), H(Y|X), and H(X)Y). Also note the ";" separator, which distinguishes mutual information from joint entropy H(X, Y). Mutual information is also often referred to in some textbooks as I(X; Y) instead of H(X; Y). Since the logarithm argument is unity when the two sources are independent (p(x,y) = p(x)p(y)), we immediately observe that the mutual information is equal to zero in this case. This reflects