# Advanced topics in information theory

(Difference between revisions)
 Revision as of 12:19, 26 June 2009 (view source) (New page: = Reading Group: Advanced Topics in Information Theory = Summer 2009 == Participants == == Topics == * Rate distortion theory (Weeks 1,2) * Network information theory (Weeks 3,4) ...)← Previous diff Revision as of 17:25, 4 July 2009 (view source) (→Topics)Next diff → Line 9: Line 9: == Topics == == Topics == - * Rate distortion theory (Weeks 1,2) + * Rate distortion theory + * Network information theory + * Kolmogorov complexity + * Quantum information theory - * Network information theory (Weeks 3,4) + == Sessions == - * Kolmogorov complexity (Weeks 5,6) + === July 7: Organization. Recap of CS-683 === + * Basic organization, presentation assignments. - * Quantum information theory (Weeks 7,8) + * Review of Information theory ideas + + * Entropy, AEP, Compression and Capacity + + Entropy of a random variable is given by $H(X) = -\sum_x p(x) \log p(x).$ + + The capacity of a channel is defined by $C = \max_{p(x)} I(X; Y).$ + + Compression and Capacity determine the two fundamental information theoretic limits of data transmission, + $H \leq R \leq C.$ + + * A review of Gaussain channels and their capacities. + + * Let us take these analysis one step further. How much do you loose when you cross these barriers? + + + === July 14:        Rate distortion theory - I === + === July 21:        Rate distortion theory - II === + === July 28:        Network Information theory- I === + === Aug 04:        Network Information theory- II === + === Aug 11:        Wireless networks, cognitive radios === + === Aug 18:        Multiple access channels, network coding techniques ===

Summer 2009

## Topics

• Rate distortion theory
• Network information theory
• Kolmogorov complexity
• Quantum information theory

## Sessions

### July 7: Organization. Recap of CS-683

• Basic organization, presentation assignments.
• Review of Information theory ideas
• Entropy, AEP, Compression and Capacity

Entropy of a random variable is given by

 H(X) = − ∑ p(x)logp(x). x

The capacity of a channel is defined by C = maxp(x)I(X;Y).

Compression and Capacity determine the two fundamental information theoretic limits of data transmission, $H \leq R \leq C.$

• A review of Gaussain channels and their capacities.
• Let us take these analysis one step further. How much do you loose when you cross these barriers?
{"ft_lang_label":"__label__en","ft_lang_prob":0.77611184,"math_prob":0.9919821,"size":2303,"snap":"2020-10-2020-16","text_gpt3_token_len":604,"char_repetition_ratio":0.16441931,"word_repetition_ratio":0.23097113,"special_character_ratio":0.3048198,"punctuation_ratio":0.12912089,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9956399,"pos_list":[0,1,2],"im_url_duplicate_count":[null,2,null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-02-17T06:31:14Z\",\"WARC-Record-ID\":\"<urn:uuid:da7373cc-7b54-4696-958b-7d903a0c4967>\",\"Content-Length\":\"24799\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:fefadb27-cc0c-4ed1-a027-dcf798153e4c>\",\"WARC-Concurrent-To\":\"<urn:uuid:10e9beab-487b-4dba-b83b-d1cfdcb88d5f>\",\"WARC-IP-Address\":\"203.135.62.52\",\"WARC-Target-URI\":\"https://cyphynets.lums.edu.pk/index.php?title=Advanced_topics_in_information_theory&diff=prev&oldid=263\",\"WARC-Payload-Digest\":\"sha1:NIJ4FO6YIMRDKCV5ZON7EJC7BGXC7WY4\",\"WARC-Block-Digest\":\"sha1:4XD4HNL2KOS2HUABXGIDURLGZBG7LZ5K\",\"WARC-Identified-Payload-Type\":\"application/xhtml+xml\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-10/CC-MAIN-2020-10_segments_1581875141749.3_warc_CC-MAIN-20200217055517-20200217085517-00453.warc.gz\"}"}