nd rainy) = p(x)p(y). (5.1) In Eq. (5.1), the function p(x, y) is called the joint probability or joint distribution of events (x,y). The meaning of the "," in the argument is that of the logical AND. The joint distribution is, thus, always symmetrical, i.e., p(x, y) = p(y, x), because the joint events x AND y and y AND X are strictly same. Since p(x, y) is a probability, we have the summing properties p(x,y) = p(y) (5.2) xeXyeY But the relation concerning any event pairs (x, y) or (z, w) of the above-defined sets is not at all this straightforward. This is because we should expect that these events are not independent: good or bad quarterly results do affect the stock market, good or bad weather forecasts do affect the outbound city traffic. One then defines the conditional probability p(a)b) as representing the probability of observing event a given the observation or knowledge of event b. If p(a)b) = p(a), this means that event a occurs regardless of b, or that the two are independent. Of Joint probabilitiesp(x, can also be written in the logical form p(x y) where the sign ^ stands for the logical or Boolean operation AND. 5.1 Joint and conditional entropies course, the same conclusion applies if p(b|a) = p(b). As we have seen in Chapter 1, the fundamental relation between joint and conditional probabilities, is given by Bayes's theorem: p(a,b)=p(ab)p(b)=p(ba)p(a). (5.3) Using the summing properties in Eq. (5.2), we also have p(a|b)p(b)= p (a) p(b|a)p(a) = 1 p(b) (5.4) Ep(ba)p(a)=Ep(ab)p(b)=1, aEAbEB aEAbEB noting that conditional probabilities p(a)x) do not sum over x up to unity, unlike the probabilities p(x) associated with single events.2 It is straightforward to verify from the above relations that if the events a and b are independent, then p(a)b) = p(a) and p(b|a) = p(b). I provide next a numerical example of joint and conditional probabilities, which we will also use further on to introduce new entropy concepts. Stock exchange Let the first event source be three possible conclusions for the quarterly s