KSQ - 2 years ago 190
Python Question

# (Python, DataFrame): Record the average of all numbers in a column that are smaller than the n'th percentile

I have a DataFrame similar to the below and would like to create a DataFrame or series that looks more like the second table.

For example: I would find the nth percentile of column A, then take the average of all numbers in A that are less than the nth percentile.

I've used the code below to get the average and range of each column but seem to be missing something to get the conditional average.

min = df.min(axis='index')

max = df.max(axis='index')

mean = df.mean(axis = 'index')

df[df < np.percentile(df, 0.4)].mean() #this doesnt seem to work and I beliebe
gives the average of every row

Table 1

``````Date    A   B   C   D   E   F
02/10/2017  10  5   1   2   1   1
01/10/2017  10  4   9   4   3   5
30/09/2017  4   8   5   6   2   4
29/09/2017  8   2   7   9   10  5
28/09/2017  3   8   2   7   10  8
27/09/2017  7   3   8   9   9   7
26/09/2017  4   1   2   9   3   4
25/09/2017  10  1   6   6   3   5
24/09/2017  8   3   5   5   6   7
23/09/2017  7   9   5   7   1   3
22/09/2017  2   9   10  5   8   1
``````

Table 2

``````Index   Avg<40th Percentile
A   3.25
B   1.333333333
C   1.666666667
D   4
E   1.333333333
F   1.666666667
`````` djk47463

Use

``````df.where(df < df.quantile(0.4)).mean()

Date         NaN
A       3.250000
B       1.333333
C       1.666667
D       4.000000
E       1.333333
F       1.666667
``````
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download
{"ft_lang_label":"__label__en","ft_lang_prob":0.52827626,"math_prob":0.7909685,"size":1088,"snap":"2019-51-2020-05","text_gpt3_token_len":472,"char_repetition_ratio":0.14944649,"word_repetition_ratio":0.0,"special_character_ratio":0.5147059,"punctuation_ratio":0.09060403,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.9570672,"pos_list":[0,1,2,3,4],"im_url_duplicate_count":[null,null,null,1,null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-01-23T07:51:25Z\",\"WARC-Record-ID\":\"<urn:uuid:458abced-e6ef-4d27-9c15-936c74cbf8e0>\",\"Content-Length\":\"46093\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:cfcb13f9-da52-4cd2-9690-0f11f4dd5310>\",\"WARC-Concurrent-To\":\"<urn:uuid:d90db812-80b2-49de-96f1-f3756497420e>\",\"WARC-IP-Address\":\"72.52.219.43\",\"WARC-Target-URI\":\"https://codedump.io/share/cMNrmcTXmsTC/1/python-dataframe-record-the-average-of-all-numbers-in-a-column-that-are-smaller-than-the-n39th-percentile\",\"WARC-Payload-Digest\":\"sha1:5DS75IBS63F62FEHI6B4QOLY3K5M4KSK\",\"WARC-Block-Digest\":\"sha1:R7PWKDTSWBZH3WSHQHOMN7JE3PH2O4GU\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-05/CC-MAIN-2020-05_segments_1579250609478.50_warc_CC-MAIN-20200123071220-20200123100220-00313.warc.gz\"}"}