followed by the letters E, T, A, O, and N, whose occurrence probabilities decrease from 10.7% and 5.8%. The entropy calculation for this source is H = 4.065 bit/symbol. If we remove the most likely occurring space character (whose frequency is not meaningful) from the source alphabet, the entropy increases to H = 4.140 bit/symbol. The "alphabet" of symbols, as meaning here the list of distinct ways of forming characters, or voice sounds, or word prefixes, roots, and suffixes, or even full words, may yet be quite large, as the phenomenally rich Chinese and Japanese languages illustrate. 8 F. Pratt, Secret and Urgent (Indianapolis: The Bobbs-Merrill Book Company, 1942). Cited in J.C. Hancock, An Introduction to the Principles of Communication Theory (New York: McGraw Hill, 1961). Entropy sp E T A O N R I S H D L F C M U G Y P W B V K X J Q Figure 4.2 Probability distribution of English alphabetical characters, as inventoried in 1942. The source entropy with or without the space character is H = 4.065 bit/symbol and H = 4.140 bit/symbol, respectively. Figure 4.3 shows the probability distributions for English, German, French, Italian, Spanish, and Portuguese, after inventories realized in 1956 or earlier.9 For comparison purposes, the data were plotted in the decreasing-probability sequence corresponding to English (Fig. 4.2). We observe that the various distributions exhibit a fair amount of mutual correlation, together or by subgroups. Such a correlation is more apparent if we plot in ordinates the different symbol probabilities against the English data, as shown in Fig. 4.4. The two conclusions are: (a) Languages make uneven use of symbol characters (corresponding to discrete- exponential PDF); (b) The PDFs are unique to each language, albeit showing a certain degree of mutual correlation. Consider next language entropy. The source entropies, as calculated from the data in Fig. 4.3, together with the 10 most frequent characters, are listed in Table 4.2. The English source is seen to have the highest entropy (4.14