r different checksums. y2 (steady) y3 (down) p(xi,yj) p 0.300 0.500 0.200 X1 (good) 0.160 0.075 0.075 0.010 0.160 X2 (same) 0.750 0.210 0.400 0.140 0.750 X3 (bad) 0.090 0.015 0.025 0.050 0.090 0.300 0.500 0.200 1.000 y1 (up) p(y1,xi) y2 (steady) p(y2,xi) y3 (down) p(y3,Xi) p(yj,xj) 0.300 0.500 = p(y2)x))(p) 0.200 = p(y3)xi)p(x) X1 (good) 0.160 0.469 0.075 0.469 0.075 0.063 0.010 X2 (same) 0.750 0.280 0.210 0.533 0.400 0.187 0.140 X3 (bad) 0.090 0.167 0.015 0.278 0.025 0.556 0.050 0.300 0.500 0.200 1.000 X1 (good) p(x1yj)(p) (same) p(x2,yj) x3 (bad) p(x3,yj) p(xi,yj) 0.160 0.750 =p(x2|yj)p(yj) 0.090 = p(x3|yj)p(yj) y1 (up) 0.300 0.250 0.075 0.700 0.210 0.050 0.015 y2 (steady) 0.500 0.150 0.210 0.800 0.400 0.050 0.025 y3 (down) 0.200 0.010 0.015 0.700 0.250 0.050 0.300 0.750 0.090 1.000 We define the joint entropy, or the entropy H(X, Y) associated with the joint distri- bution p(x,y) with X E X and y E Y, namely: H(X,Y)=-2Xp(x,y)log2p(x,y). (5.5) xeXyeY This joint entropy represents the average information derived from joint events occurring from two sources X and Y. The unit of H(X, Y) is bit/symbol. We then define the conditional entropy H(X|Y) through: H(XY)=-22p(x,y)log2p(xly). (5.6) The conditional entropy H(X|Y) corresponds to the average information conveyed by the conditional PDF, p(x/y). Put simply, H(X|Y) represents the information we learn from source X, given the information we have from source Y. Its unit is also bit/symbol. Or, equivalently, H(X, Y) = - 5.1 Joint and conditional entropies In the joint and conditional entropy definitions, note that the two-dimensional aver- aging over the event space {X, Y} consistently involves the joint distribution p(x, y). As a property, the conditional entropy H(X|Y) is generally different from H(Y|X), which is defined as (5.7) since in the general case, p(x/y) # p(y|x). The conditional entropy H(X|Y) or H(Y|X) is sometimes referred to by the elegant term equivocation. It is easily verified that if the sources X, Y represent independent events, then (a)H(X,Y)=H(X