ormation, namely, orders of magnitude below (10-6 nat). In the definition of I(x), it is also possible to choose base-two or base-ten logarithms. This changes the information scale, but the qualitative result remains the same: infor- mation continuously increases as the likelihood of events decrease. I will come back to this issue later. Next, let us explore some more properties of the proposed information measure. Consider two unrelated or independent events A and B, with probabilities p(A) and p(B). We define the joint probability p(A, B) as the probability that both events A and B occur. As we have seen in Chapter 1, if A and B are independent events, then p(A,B) = p(A)p(B). The information associated with the joint events is, therefore, 5 = -log[p(A,B)] = -log[p(A)p(B)] = -log[p(A)]-log[p(B)] (3.6) The result shows that the information measure of the joint event is the sum of the information measures associated with each event considered separately. Information is, thus, an additive quantity. Such a property holds true only if the events are independent. The more complex case where events are not independent will be addressed later on. 5 Using the rule for logarithms: log(XY) = log X + log Y (X and Y being positive). 3.3 Information bits In the following, I will refer to information measure simply as information, in the IT meaning of the word. What if the two events A and B are not independent? Then we know from Bayes's the- orem that p(A, B) = p(A|B)p(B) = p(B|A)p(A). The information associated with the joint event is, therefore, -log[p(A,B)] -log[p(A)p(B|A)] = -log[p(A)]-log[p(B\A)] (3.7) = =((A)+I[p(B|A)]. The result shows that the information of the joint event is the sum of the information concerning event A and the information concerning event B - knowing that event A occurred. The term I[p(B|A)] is referred to as conditional information. Information bits Since we have established that information can be measured according to its degree of surprise, it is interesting to find the minimal information meas