Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

After working on the MOB package, I received requests from multiple users if I can write a binning function that takes the weighting scheme into consideration. It is a legitimate request from the practical standpoint. For instance, in the development of fraud detection models, we often would sample down non-fraud cases given an extremely low frequency of fraud instances. After the sample down, a weight value > 1 should be assigned to all non-fraud cases to reflect the fraud rate in the pre-sample data.

While accommodating the request for weighting cases is trivial, I’d like to do a simple experitment showing what the impact might be with the consideration of weighting.

– First of all, let’s apply the monotonic binning to a variable named “tot_derog”. In this unweighted binning output, KS = 18.94, IV = 0.21, and WoE values range from -0.38 to 0.64.

– In the first trial, a weight value = 5 is assigned to cases with Y = 0 and a weight value = 1 assigned to cases with Y = 1. As expected, frequency, distribution, bad_frequency, and bad_rate changed. However, KS, IV, and WoE remain identical.

– In the second trial, a weight value = 1 is assigned to cases with Y = 0 and a weight value = 5 assigned to cases with Y = 1. Once again, KS, IV, and WoE are still the same as the unweighted output.

The conclusion from this demonstrate is very clear. In cases of two-value weights assigned to the binary Y, the variable importance reflected by IV / KS and WoE values should remain identical with or without weights. However, if you are concerned about the binning distribution and the bad rate in each bin, the function wts_bin() should do the correction and is available in the project repository (https://github.com/statcompute/MonotonicBinning).
{"ft_lang_label":"__label__en","ft_lang_prob":0.91102,"math_prob":0.9781329,"size":2142,"snap":"2023-40-2023-50","text_gpt3_token_len":499,"char_repetition_ratio":0.1272217,"word_repetition_ratio":0.16304348,"special_character_ratio":0.22969188,"punctuation_ratio":0.125,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9618785,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2023-11-28T11:07:53Z\",\"WARC-Record-ID\":\"<urn:uuid:98e171c5-ffb7-45cf-bd21-41c1df98f6a3>\",\"Content-Length\":\"84915\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:0fa33ba9-21a5-4a04-9bda-95b320e995d1>\",\"WARC-Concurrent-To\":\"<urn:uuid:6d7e88f4-9766-4f6f-9b29-301977f00bec>\",\"WARC-IP-Address\":\"104.21.77.221\",\"WARC-Target-URI\":\"https://www.r-bloggers.com/2019/04/binning-with-weights/\",\"WARC-Payload-Digest\":\"sha1:ZRCKKUXYA5ZXE3Z4FRA7JBRWOEYVOHH3\",\"WARC-Block-Digest\":\"sha1:M7HLSLJDJJEEEHE3CYBGDF7K3FY5WMWL\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2023/CC-MAIN-2023-50/CC-MAIN-2023-50_segments_1700679099281.67_warc_CC-MAIN-20231128083443-20231128113443-00442.warc.gz\"}"}