# Calibrating a multi-class boosted classifier

I have read Alexandru Niculescu-Mizil and Rich Caruana's paper "Obtaining Calibrated Probabilities from Boosting" and the discussion in this thread. However, I am still having trouble understanding and implementing logistic or Platt's scaling to calibrate the output of my multi-class boosting classifier (gentle-boost with decision stumps).

I am somewhat familiar with generalized linear models, and I think I understand how the logistic and Platt's calibration methods work in the binary case, but am not sure I know how to extend the method described in the paper to the multi-class case.

The classifier I am using outputs the following:

• $f_{ij}$ = Number of votes that the classifier casts for class $j$ for the sample $i$ that is being classified
• $y_i$ = Estimated class

At this point I have the following questions:

Q1: Do I need to use a multinomial logit to estimate probabilities? or can I still do this with logistic regression (e.g. in a 1-vs-all fashion)?

Q2: How should I define the intermediate target variables (e.g. as in Platt's scaling) for the multi-class case?

Q3: I understand this might be a lot to ask, but would anybody be willing to sketch out the pseudo-code for this problem? (on a more practical level, I am interested in a solution in Matlab).

• great question. I have wondered as well about how to construct the calibration even if you do use 1 versus the rest sort of scheme. If you create k models using 1 versus the rest (and there are k classes) do you have to / should you normalize them somehow so that they sum to 1 (e.g. divide each calibrated probability by the sum of all k)? – B_Miner Jan 28 '11 at 1:10
{"ft_lang_label":"__label__en","ft_lang_prob":0.92570156,"math_prob":0.58361846,"size":1276,"snap":"2019-51-2020-05","text_gpt3_token_len":287,"char_repetition_ratio":0.1014151,"word_repetition_ratio":0.0,"special_character_ratio":0.21551724,"punctuation_ratio":0.09128631,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9946032,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2019-12-14T20:57:55Z\",\"WARC-Record-ID\":\"<urn:uuid:bfc5762a-8f7a-4aac-a872-dee0f5149cb8>\",\"Content-Length\":\"136860\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:e5f5db22-e8e4-4cb8-a033-d641a6451603>\",\"WARC-Concurrent-To\":\"<urn:uuid:462967c7-510d-46d1-b1a8-f022596cbc46>\",\"WARC-IP-Address\":\"151.101.1.69\",\"WARC-Target-URI\":\"https://stats.stackexchange.com/questions/6599/calibrating-a-multi-class-boosted-classifier\",\"WARC-Payload-Digest\":\"sha1:FZIKGH7722HUJXMRSCF7CPNKHTEB3CE6\",\"WARC-Block-Digest\":\"sha1:BGIT45HM7P3ZAJPHBMFT34Z5GIUU2DN4\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2019/CC-MAIN-2019-51/CC-MAIN-2019-51_segments_1575541294513.54_warc_CC-MAIN-20191214202754-20191214230754-00488.warc.gz\"}"}