ceptual subtlety, the word entropy has always been enshrouded in some mystery, even, as today, to large and educated audiences. The subsequent works of L. Boltzmann, which set the grounds of statistical mechan- ics, made it possible to provide further clarifications of the definition of entropy, as a natural measure of disorder. The precursors and founders of the later information theory (L. Szilárd, H. Nyquist, R. Hartley, J. von Neumann, C. Shannon, E. Jaynes, and L. Brillouin) drew as many parallels between the measure of information (the uncer- tainty in communication-source messages) and physical entropy (the disorder or chaos within material systems). Comparing information with disorder is not at all intuitive. This is because information (as we conceive it) is pretty much the conceptual opposite of disorder! Even more striking is the fact that the respective formulations for entropy that have been successively made in physics and IT happen to match exactly. A legend has it that Shannon chose the word "entropy" from the following advice of his colleague von Neumann: "Call it entropy. No one knows what entropy is, SO if you call it that you will win any argument." This chapter will give us the opportunity to familiarize ourselves with the concept of entropy and its multiple variants. So as not to miss the nice parallel with physics, we will start first with Boltzmann's precursor definition, then move to Shannon's definition, and develop the concept from there. From Boltzmann to Shannon The derivation of physical entropy is based on Boltzmann's work on statistical mechanics. Put simply, statistical mechanics is the study of physical systems made of large groups of This choice could also be attributed to the phonetic similarity with the German Energie (energy, or energia in Greek), SO the word can also be interpreted as "energy turning point" or "point of energy transformation." For a basic definition of the second law of thermodynamics, see, for instance, http://en.wikipedia.org/ wiki/Entropy. 4.1 From Boltzm