# Categorical Cross Entropy

These are some additional notes that I am taking on the incredible book by David Foster on Generative Deep Learning The calculation for this is:

$$\Huge - \overset{n}{\underset{i = 1}{\Sigma}} y_i\space log(p_i)$$

This simply shows the log of the prediction times the ground truth. Then you sum all of this up for each data point.
{"ft_lang_label":"__label__en","ft_lang_prob":0.8567255,"math_prob":0.9798796,"size":478,"snap":"2021-21-2021-25","text_gpt3_token_len":115,"char_repetition_ratio":0.06962025,"word_repetition_ratio":0.0,"special_character_ratio":0.22803347,"punctuation_ratio":0.06666667,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9537872,"pos_list":[0,1,2],"im_url_duplicate_count":[null,null,null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2021-06-15T12:51:21Z\",\"WARC-Record-ID\":\"<urn:uuid:6c04e040-23e4-4291-87c0-6ff979aa85e7>\",\"Content-Length\":\"8681\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:ab87c72b-6722-40e8-90f6-4c38e8e0c567>\",\"WARC-Concurrent-To\":\"<urn:uuid:fd27c59d-12b5-4e31-978d-e094088f4809>\",\"WARC-IP-Address\":\"185.199.111.153\",\"WARC-Target-URI\":\"https://ralphabrooks.com/deep_learning/dictionary/categoricalcrossentropy/\",\"WARC-Payload-Digest\":\"sha1:56M4SS72EYWKLROEI6OV7VNOGGYZSXEU\",\"WARC-Block-Digest\":\"sha1:UG7QVZJWL7MX3KDWT54VQJDGEWXA3KF4\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2021/CC-MAIN-2021-25/CC-MAIN-2021-25_segments_1623487621273.31_warc_CC-MAIN-20210615114909-20210615144909-00113.warc.gz\"}"}