so that and so and and so by Kraft's inequality there exists a prefix-free code having those word lengths. Thus the minimal satisfies Extension to non-stationary independent sources Fixed Rate lossless source coding for discrete time non-stationary independent sources Define typical set as: Then, for given , for large enough, . Now we just encode the sequences in the typical set, and usual methods in source coding show that the cardinality of this set is smaller than . Thus, on an average, bits suffice for encoding with probability greater than , where and can be made arbitrarily small, by making larger. See also Channel coding Noisy-channel coding theorem Error exponent Asymptotic Equipartition Property (AEP) References Information theory Coding theory Data compression Presentation layer protocols Mathematical theorems in theoretical computer science Articles containing proofs