ann to Shannon Energy Nm-1 1 N3= = 4 OO--O--O--O-O-O--O N1=8 = Number of particles Nj Figure 4.1 Energy-level diagram showing how a set of N identical particles in a physical macrosystem can be distributed to occupy, by subsets of number N, different microstates of energy E (i = 1 ...m). particles, for which it is possible to assign both microscopic (individual) and macroscopic (collective) properties. Consider a macroscopic physical system, which we refer to for short as a macrosystem. Assume that it is made of N particles. Each individual particle is allowed to occupy one out of m possible microscopic states, or microstates,3 which are characterized by an energy E (i = 1...m), as illustrated in Fig. 4.1. Calling N the number of particles occupying, or "populating," the microstate of energy Ei, the total number of particles in the macroscopic system is (4.1) and the total macrosystem energy is E=ENE. (4.2) Now let's perform some combinatorics. We have N particles each with m possible energy states, with each state having a population N. The number of ways W to arrange the N particles into these m boxes of populations N is given by: (4.3) 3 A microscopic state, or microstate, is defined by a unique position to be occupied by a particle at atomic scale, out of several possibilities, namely, constituting a discrete set of energy levels. Entropy (see Appendix A for a detailed demonstration). When the number of particles N is large, we obtain the following limit (Appendix A): (4.4) where Pi = N/N represents the probability of finding the particle in the microstate of energy E. As formulated, the limit H can thus be interpreted as representing the average value (or "expectation value" or "statistical mean") of the quantity -log Pi, namely, H = -(logp). This result became the Boltzmann theorem, whose author called H "entropy." Now let us move to Shannon's definition of entropy. In his landmark paper, Shannon seeks for an improved and comprehensive definition of information measure, which he called H. In the following, 