is well-defined. We can use Boltzmann's equation for the entropy change as applied to the mixing process where is the Boltzmann constant. We then calculate the number of ways of arranging molecules of component 1 and molecules of component 2 on a lattice, where is the total number of molecules, and therefore the number of lattice sites. Calculating the number of permutations of objects, correcting for the fact that of them are identical to one another, and likewise for , After applying Stirling's approximation for the factorial of a large integer m: , the result is where we have introduced the mole fractions, which are also the probabilities of finding any particular component in a given lattice site. Since the Boltzmann constant , where is the Avogadro constant, and the number of molecules , we recover the thermodynamic expression for the mixing of two ideal gases, This expression can be generalized to a mixture of components, , with Relationship to information theory The entropy of mixing is also proportional to the Shannon entropy or compositional uncertainty of information theory, which is defined without requiring Stirling's approximation. Claude Shannon introduced this expression for use in information theory, but similar formulas can be found as far back as the work of Ludwig Boltzmann and J. Willard Gibbs. The Shannon uncertainty is not the same as the Heisenberg uncertainty principle in quantum mechanics which is based on variance. The Shannon entropy is defined as: where pi is the probability that an information source will produce the ith symbol from an r-symbol alphabet and is independent of previous symbols. (thus i runs from 1 to r ). H is then a measure of the expected amount of information (log pi) missing before the symbol is known or measured, or, alternatively, the expected amount of information supplied when the symbol becomes known. The set of messages of length N symbols from the source will then have an entropy of NH. The thermodynamic entropy is only due to positional uncertainty, so we may take the "alphabet" to be any of the r different species in the gas, and, at equilibrium, the probability that a given particle is of type i is simply the mole fraction xi for that particle. Since we are dealing with ideal gases, the identity of nearby particles is irrelevant. Multiplying by the number of particles N yields the change in entropy of