and crossing out terms appearing in both numerator and denominator, two by two, leads to the result in Eq. (A1). Appendix A Task 2 Show that (1/N) log W takes the following limit when the number of particles N (N = -1 Ni) becomes infinite: where and Pi = Ni/N is the probability of finding N particles in the microstate of energy E. To demonstrate the above result, notice first that from Eq. (A1), we have Assuming x = N, N, is large, we use Stirling's approximation formula: where the series in parenthesis can be approximated to unity. Thus, Boltzmann's entropy = log N N log (2) log N 1 EN N + = pilogpi. (A10) Appendix B (Chapter 4) Shannon's entropy1 Consider a source with N elements of occurrence probability pi i (i = 1...N). We look for an information measure H, which should meet three conditions: (1) H(p1, P2, PN) is a continuous function of the probability set Pi; (2) If all probabilities were equal (namely pi = 1/N), the function H should increase monotonously with N; (3) If any occurrence breaks down into two successive possibilities, the original H should break down into a weighed sum of the corresponding individual values of H. Step 1 We first define A(N).1),which is = the value taken by H when prob- abilities are equal. Consider now A(2) = H(1,1), which is the mean information of two equiprobable events. After condition (3), if the second event represents two equal possibilities, the new information measure is given by the weighted sum The first term in the left-hand side represents the information contribution of the two events, and the second term represents the extra information contained in the second event (which occurs half of the time). If the first event also represents two equal possi- bilities, we have A(4) =2A(2) A(22)=2A(2), This is a more detailed description of Shannon's demonstration. See C.E. Shannon, A mathematical theory of communication. Bell Syst. Tech. J., 27 (1948), 79-423, 623-56. This paper can be downloaded from http://cm.bell-labs.com/cm/ms/what/shannonday/paper.html Shannon's ent