title: Creating a new column from two columns using a dictionary in Pandas
body: <p>I want to create a column based on a group and threshold for cutoff from another column for each group of the grouped column.</p>
  
  <p>The dataframe is below:</p>
  
  <pre><code>     df_in -&gt; 
  
         unique_id  myvalue identif
     0      CTA15   19.0     TOP
     1      CTA15   22.0     TOP
     2      CTA15   28.0     TOP
     3      CTA15   18.0     TOP
     4      CTA15   22.4     TOP
     5      AC007    2.0     TOP
     6      AC007    2.3    SDME
     7      AC007    2.0    SDME
     8      AC007    5.0    SDME
     9      AC007    3.0    SDME
     10     AC007   31.4    SDME
     11     AC007    4.4    SDME
     12      CGT6    9.7    BTME
     13      CGT6   44.5    BTME
     14      TVF5    6.7    BTME
     15      TVF5    9.1    BTME
     16      TVF5   10.0    BTME
     17      BGD1    1.0    BTME
     18      BGD1    1.6     NON
     19       GHB   51.0     NON
     20       GHB   54.0     NON
     21       GHB    4.7     NON
  </code></pre>
  
  <p>So I have created a dictionary based on each group of the 'identif' column as :</p>
  
  <pre><code>    md = {'TOP': 22, 'SDME': 10, 'BTME': 20, 'NON':20}
  </code></pre>
  
  <p>So my goal is to create a new column, say 'chk', based on the following condition:</p>
  
  <p>If the "identif" column matches the key in the dictionary "md" and the value for that key is >= than the corresponding value in the "myvalue" column then 
  I will have 1, otherwise 0. </p>
  
  <p>However, I am trying to find a good way using map/groupby/apply to create the new output data frame. I am now doing a very inefficient way ( which is taking considerable time on real data of million rows) 
  using a function as follows: </p>
  
  <pre><code> def myfilter(df, idCol, valCol, mydict):
  
     for index,row in df.iterrows():
        for key, value in mydict.items(): 
           if row[idCol] == key and row[valCol] &gt;= value:
               df['chk'] = 1
           elif row[idCol] == key and row[valCol] &lt; value:
               df['chk'] = 0
  
    return df
  </code></pre>
  
  <p>Getting the output via the following call:</p>
  
  <pre><code>     df_out = myfilter(df_in, 'identif', 'myvalue', md)
  </code></pre>
  
  <p>So my output will be like: </p>
  
  <pre><code>     df_out -&gt;
  
           unique_id  myvalue identif  chk
      0      CTA15     19.0     TOP    0
      1      CTA15     22.0     TOP    1
      2      CTA15     28.0     TOP    1
      3      CTA15     18.0     TOP    0
      4      CTA15     22.4     TOP    1
      5      AC007      2.0     TOP    0
      6      AC007      2.3    SDME    0
      7      AC007      2.0    SDME    0
      8      AC007      5.0    SDME    0
      9      AC007      3.0    SDME    0
      10     AC007     31.4    SDME    1
      11     AC007      4.4    SDME    0
      12      CGT6      9.7    BTME    0
      13      CGT6     44.5    BTME    1
      14      TVF5      6.7    BTME    0
      15      TVF5      9.1    BTME    0
      16      TVF5     10.0    BTME    0
      17      BGD1      1.0    BTME    0
      18      BGD1      1.6     NON    0
      19       GHB     51.0     NON    1
      20       GHB     54.0     NON    1
      21       GHB      4.7     NON    0
  </code></pre>
  
  <p>This works but extremely inefficient and would like a much better way to do it.</p>