title: Pandas apply: multiple conditions and multiple function arguments
body: <p>In a nutshell, I am trying to apply my function to select row. I have it working by subsetting the dataframe, run the function and then merge the subset back to the main dataframe. However, that is cumbersome and there has to be more efficient solution that escapes me. I found several useful posts (<a href="https://stackoverflow.com/questions/36909977/update-row-values-where-certain-condition-is-met-in-pandas">here</a>, <a href="https://stackoverflow.com/questions/48872234/using-apply-in-pandas-lambda-functions-with-multiple-if-statements">here</a> and <a href="https://stackoverflow.com/questions/12182744/python-pandas-apply-a-function-with-arguments-to-a-series">here</a>) that helped improve my code. </p>
  
  <p>Here is a sample dataframe:</p>
  
  <pre><code>data = {'firm': ['Smith', 'Jones', 'Smith New York', 'Jones International', 'Winter'], 
          'id': [np.nan, 732, 216, np.nan, 1714], 
          'url1': ['url', np.nan, 'url', 'url', 'url'],
          'url2': ['url', 'url', 'url', np.nan, 'url'],
          'text': ['foo', 'bar', np.nan, np.nan, 'foo bar']}
  df = pd.DataFrame(data)
  </code></pre>
  
  <p>The below function will parse the website whereby the user can set the keyword to search already downloaded files and use that stored data if present. If the last crawl happened a while a go a new crawl for an updated website is needed.</p>
  
  <pre><code>def fetch(id, url, **kwargs):
      if backup == 'Yes':
          print('Fetching {} from {}'.format(id, url))
          # Actual fetching code
      else:
          print('Loading stored data for {}'.format(id))
          # Actual loading code 
  </code></pre>
  
  <p>The function works as I tested it on individual URLs, but I run into problems when I try to apply it. I have multiple conditions when to run it. Currently I use them to subset the dataframe.  Note: if two urls are present, url1 is preferred. Following <a href="http://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.apply.html" rel="nofollow noreferrer">Pandas documentation</a> keyword arguments can be submitted. Initially I tried <code>np.where</code>. There are 4 conditions in total, below are two:</p>
  
  <pre><code>df['content'] = np.where(df['text'].isna() &amp; df['url1'].notnull() &amp;
                              df['url2'].notnull() &amp; df['firm'].str.contains('Smith'),
                           df['url1'].apply(fetch, args=df['id'], backup='Yes'),
                           np.where(df['text'].isna() &amp; df['url1'].notnull() &amp; 
                                      df['url2'].isna() &amp; df['firm'].str.contains('Smith'),
                                    df['url1'].apply(**fetch, backup='Yes'**),
                                    pd.np.nan))
  TypeError: fetch() takes 2 positional arguments but --some other number-- were given
  </code></pre>
  
  <p>Hence, adding pandas series does not work. And I cannot figure out how to add it as a scalar.
  Another failed approach with only two of the columns/series:</p>
  
  <pre><code>df[['id', 'url1']][fd['text'].isna() &amp; df['url1'].notnull() &amp;
      df['url2'].notnull() &amp; df['firm'].str.contains('Smith')].apply(fetch) # Should fetch nothing
  df[['id', 'url1']][fd['text'].isna() &amp; df['url1'].notnull() &amp;
      df['url2'].isna() &amp; df['firm'].str.contains('Smith')].apply(fetch) # Should fetch one
  TypeError: ("fetch() missing 1 required positional argument: 'url1'", 'occurred at index id')
  </code></pre>
  
  <p>And finally I tried <code>lambda</code>:</p>
  
  <pre><code>df['text'].where(fd['text'].isna() &amp; df['url1'].notnull() &amp; df['url2'].isna()
     &amp; df['fidm'].str.contains('Smith'), df[['id', 'url1']].apply(lambda x,y: get_XML(x,y)))
  TypeError: ("&lt;lambda&gt;() missing 1 required positional argument: 'y'", 'occurred at index id')
  </code></pre>
  
  <p>I assume I am missing something simple, but obviously crucial. Any pointers are appreciated. </p>
  
  <hr>
  
  <p><strong>Edit - Solution</strong></p>
  
  <hr>
  
  <p>I took comments from <a href="https://stackoverflow.com/users/119603/damien-ayers">Damien Ayers</a> (see below) to heart and simplified the code. This then also put me on the path to the solution:</p>
  
  <pre><code>def get_ft(text, xml, id, url1, url2, firm, backup= 'Yes'):
      if pd.notnull(id):
          if pd.isna(text) and pd.notnull(url1) and (pd.notnull(url2) or pd.isna(url2)):
              if 'Smith' in firm:
                  return fetch(id, url1, backup)
              ... code continues
  </code></pre>
  
  <p>And here the proper use of <code>apply</code> and <code>lambda</code> thanks to <a href="https://stackoverflow.com/questions/34279378/python-pandas-apply-function-with-two-arguments-to-columns">this discussion</a>:</p>
  
  <pre><code>df['text_new'] = df.apply(lambda x: x['text'], x['id'], x['url1'],
                                      x['url2'], x['firm'], backup), axis=1)
  </code></pre>
  
  <p>Much cleaner and more importantly it works. </p>