# Work log 2016-07-13

Posted on by Chris Warburton

We could learn a compact representation of a generative model; already dont with e.g. graphs.

For each symbol S(i) assign probabilities P(i, 0), P(i, 1), etc.

Initial model: P(S(i)) = P(i, 0)

Expand:

``````λv(i) -> <term>
<term> <term>
case <term> of
<pat> -> <term>
...``````

Not the brillest; how about Church Encoding?

~~ `λs0 s1 s2 ... -> <term>` ~~

For n symbols:

``````term n = λ s1 s2 ... sn -> <expr n>

expr n = vi | 0 <= 1 < n
| λ -> <expr (S n)>
| <expr n> <expr n>``````

What can we compare recurrent clustering to?

• Non-recurrent clustering
• Simpler feature extraction/distance, e.g. n-grams, bag-of-words, tree kernel

Ranking would be good too; select first N closest matches.

We want to learn a rule from 𝒫(Def) -> 𝒫(Def) (in general). Maybe more practical for 𝒫(Def) -> Def -> [0,1]
{"ft_lang_label":"__label__en","ft_lang_prob":0.69135875,"math_prob":0.9607675,"size":822,"snap":"2022-27-2022-33","text_gpt3_token_len":273,"char_repetition_ratio":0.10880195,"word_repetition_ratio":0.0,"special_character_ratio":0.35523114,"punctuation_ratio":0.18478261,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9873794,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2022-08-18T19:49:18Z\",\"WARC-Record-ID\":\"<urn:uuid:fa1677cf-c8c1-4862-9265-dd1d7335f092>\",\"Content-Length\":\"2753\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:dd4d691c-f29c-45fd-8236-90f6c8080c15>\",\"WARC-Concurrent-To\":\"<urn:uuid:5fe7e871-ae36-4d7f-963a-546c11997ba2>\",\"WARC-IP-Address\":\"52.92.0.140\",\"WARC-Target-URI\":\"http://www.chriswarbo.net/blog/2016-07-13-work_log.html\",\"WARC-Payload-Digest\":\"sha1:OSQNPA2VBFZXQX4XKRRI4BFVX7PNXGQA\",\"WARC-Block-Digest\":\"sha1:XS2RBKHK2WOE4WQLWD6NV3C74MVM4HHJ\",\"WARC-Identified-Payload-Type\":\"application/xhtml+xml\",\"warc_filename\":\"/cc_download/warc_2022/CC-MAIN-2022-33/CC-MAIN-2022-33_segments_1659882573399.40_warc_CC-MAIN-20220818185216-20220818215216-00583.warc.gz\"}"}