concepts here. In the case of two ensembles, there exist four subset possibilities, as defined by their elements's properties: Elements common to A or B: AUB, Elements common to A and B: ANB, Elements from A and not B: An-B, Elements from B and not A: BN-A. In the conventional notations shown at right, we see that the symbol U stands for a logical OR, the symbol n stands for a logical AND, and the symbol - stands for a logical NO. These three different symbols, which are also called Boolean operators, make it possible to perform various mathematical computations in the field called Boolean logic. In the case of three ensembles A, B, C, we observe that there exist many more subset possibilities (e.g., AnBnC, ANBN-C); it is left as an exercise to the reader to enumerate and formalize them in terms of Boolean expressions. The interest of the above visual description with the Venn diagrams is the straightforward correspondence with the various entropy definitions that have been introduced. Indeed, it can be shown 4 To be accurate, Boolean logic uses instead the symbol V for "or" (the equivalent of U in ensemble language), and the symbol ^ for "and" (the equivalent of n in ensemble language). 5.2 Mutual information H(X;Y) H(X,Y) H(YX) H(XY) H(X;YZ) H(Y),Z,Z) H(X,Y,Z) H(XY,Z) H(X;Y;Z) H(ZXXY) Figure 5.2 Venn diagram representation of entropy H(U), joint entropy H(U,V), conditional entropy H(UV), and mutual information H(U;V), for two (U = X, Y) or three (U,V = X,Y,Z) sources. that the following equivalences hold: H(X, Y) H(XUY) H(X; H(XOY) (5.15) H(X|Y) H(Xn-Y) H(Y|X) H(YO-X). The first equivalence in Eq. (5.15) means that the joint entropy of two sources is the entropy of the source defined by their combined events. The second equivalence in Eq. (5.15) means that the mutual information of two sources is the entropy of the source containing the events they have in common. The last two equivalences in Eq. (5.15) provide the definition of the conditional entropy of a source U given the information on a source V. The cond