those plete knowledge. Mutual and self information are introduced and holding these, and other views. However, these views just reflect the these concepts are applied to various models for communications particular problems that were being addressed by the authors, and channels. one should take the view that the above are valid and useful in The concepts of Information and the related quantity Entropy, date different contexts, but that some views are more general than others, back to the early days of thermodynamics. The concept of entropy and it is this general approach that will now be discussed before we arose in considerations connected with the theory of heat, and its apply it to information theory. relationship to information was not at first realised. However, in According to the view put forward by Jeffreys, probability ex- retrospect it is not surprising that the analytical treatment of infor- presses a state of knowledge about any system under investigation, mation should have its origin in this area of physical science, for the and as such applies to a very wide range of problems. This approach subject of thermodynamics is primarily concerned with the determi- to probability theory incorporates many of the above mentioned nation of the laws that govern the conversion of mechanical and interpretations. other forms of energy, into heat energy. Heat is characterised by its It should not be considered wrong to adopt a narrow view of disorder, and ultimately by the irreversibility of certain processes probabilities in terms of random experiments and frequencies of which involve heat transfer. The degree of organisation of a system occurrence, but there is certainly nothing to be lost in adopting a can also be interpreted as a measure of the quantity of information broader view in terms of states of knowledge, and if the problem incorporated in it. This idea, was developed by Szilard in 1929, after requires the concept of relative frequencies etc., this should emerge which time the notion of entropy was anot