)+H(Y), (5.8) (b) H(X)Y) = H(X)m and H(Y|X) = H(Y). (5.9) As it can also be easily established, the joint and conditional entropy are related to each other through the chain rule: H(X,Y)=H(XY)+H(Y) (5.10) H(X, Y) = H(Y|X) + H(X). We may find the chain rule easier to memorize under the form H(X|Y) H(X, Y) - H(Y) (5.11) H(Y|X) H(X, Y) - H(X), which states that, given a source X or Y, any advance knowledge (or "conditioning") from the other source Y or X reduces the joint entropy "reserve" H(X, Y) by the net amount H(Y) or H(X), respectively; these are positive quantities. In other words, the prior information one may gain from a given source is made at the expense of the information available from the other source, unless the two are independent. We can illustrate the above properties through our stock-exchange PDF data from Table 5.1. We want, here, to determine how the average information from the com- pany's sales, H(X), is affected from that concerning the stocks, H(Y), and the reverse. The computations of H(X), H(Y), H(X, Y), H(Y|X) and H(X)Y) are detailed in Table 5.2. It is seen from the table that the results and stock entropies compute to H(X) = 1.046 and H(Y) = 1.485, respectively (for easier reading, I omit here the bit/symbol units). The joint entropy is found to be H(X, Y) = 2.466, which is lower than the sum H(X) + H(Y) = 2.532. This proves that the two sources are not independent, namely, that they have some information in common. We find indeed that the conditional entropies satisfy: H(YX)=1.418<H(Y)=1.485, = = H(X|Y) = =0.980<H(X)= 1.046, or, equivalently, (using four decimal places, for accuracy (see Table 5.2): H(Y) - H(Y|X) = 1.4855-1.4188 =0.0667, H(X) - H(X|Y) = 1.0469 - 0.9802 = 0.0667. Mutual information and more entropies Table 5.2 From top to bottom: calculation of entropies H(X), H(Y), H(X, Y), H(Y)X) and H(X)Y), as based on the numerical example of Table 5.1. The two equations shown at the bottom of the table prove the fundamental relations between single-event entropies, conditional ent