# How to set a threshold on softmax probabilities in a multi-class classification task?

I have a large image dataset that was classified by a ConvNet into different classes (objects). For each image the top-1 softmax probability is given, ranging between 0 and 1. It´s the output of a multi-class classification task, so the softmax classification output contains multiple values, for example (0.6, 0.1, 0.2, 0.1). The top-1 probability, in this example, would be 0.6. In my dataset the top-1 softmax probability of many images is rather low (e.g. 0.1), meaning that the probability that the image shows the predicted class is low. Now I am wondering if and how I should set a threshold on the softmax probabilities. My approach was to compare the predicted labels with ground-truth labels (which are available for ca. 10% of the whole dataset), plot a ROC curve and calculate the Youden Index and optimum cut-off point. Then I used this optimum cut-off point as a threshold for the softmax probabilities and removed all images from the dataset with a top-1 softmax probability below this cut-off point. This reduced my dataset to ~1/4 of its original size.

My questions are: - Can I use the described approach to define a threshold for the top-1 softmax probabilities? - Are there other approaches, for example defining a threshold for each class? And how one would do this?
{"ft_lang_label":"__label__en","ft_lang_prob":0.90929574,"math_prob":0.9275446,"size":1287,"snap":"2020-45-2020-50","text_gpt3_token_len":291,"char_repetition_ratio":0.15042868,"word_repetition_ratio":0.0,"special_character_ratio":0.22999223,"punctuation_ratio":0.123595506,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.99524856,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-11-27T17:19:45Z\",\"WARC-Record-ID\":\"<urn:uuid:c457bc2e-7489-495f-aaf5-c3dbec6726be>\",\"Content-Length\":\"153304\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:6da5b95f-6fb0-47dd-b8b0-1bc6c710a077>\",\"WARC-Concurrent-To\":\"<urn:uuid:3e367704-507a-4e80-adca-6dd1350f22ac>\",\"WARC-IP-Address\":\"151.101.1.69\",\"WARC-Target-URI\":\"https://stats.stackexchange.com/questions/471058/how-to-set-a-threshold-on-softmax-probabilities-in-a-multi-class-classification\",\"WARC-Payload-Digest\":\"sha1:EOSQHPBCASOE7XEIFI6WKWISVS2RU4FE\",\"WARC-Block-Digest\":\"sha1:DOLZP4AYYIHTLX5MGVL7APJM26ONX5BF\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-50/CC-MAIN-2020-50_segments_1606141193856.40_warc_CC-MAIN-20201127161801-20201127191801-00718.warc.gz\"}"}