Title: How do i change multiple columns of specific rows in pandas dataframe with custom functions?
Tags: python;pandas;dataframe
Question: I would like to edit the following dataframe with custom function defined, my dataframe returned empty for rows that passed through the custom function
field      value1            value2          value3
0    target     ['list']       ['123','length']     'abc'
1    random      [32.67]           [47.00]          'bcd'
2    target      ['word']           [105]           'efg'
3    target2   [&quot;key&quot;:&quot;123&quot;]    [&quot;key&quot;:&quot;333&quot;]       'xyz'
my custom function
def f(field, value1, value2):
if field == 'target':
value1 = [{item} for item in value1]
value2 = [{item} for item in value2]
elif field == 'target2':
value1 = [{item.values()} for item in value1]
value2 = [{item.values()} for item in value2]
return pd.Series([value1, value2])
so i did the following
mask = df['field'].isin(['target','target2'])
df.loc[mask,['value1','value2']] = df[mask].apply(lamba row: f(row['field'], row['value1'], row['value2'], axis=1)
my result is the following, which caused my targetted row to return as NaN
field      value1       value2     value3
0    target       NaN          NaN        'abc'
1    random      [32.67]      [47.00]     'bcd'
2    target       NaN          NaN        'efg'
3    target2      NaN          NaN        'xyz'
my Expected result
field      value1               value2              value3
0    target     [{'list'}]       [{'123'},{'length'}]     'abc'
1    random      [32.67]             [47.00]              'bcd'
2    target      [{'word'}]          [{105}]              'efg'
3    target2    [{&quot;123&quot;}]            [{&quot;333&quot;}]            'xyz'
Thank you!


Here is another answer: You need set the index of returned Series to matched the columns
def f(field, value1, value2):
if field == 'target':
value1 = [{item} for item in value1]
value2 = [{item} for item in value2]
elif field == 'target2':
value1 = [{item.values()} for item in value1]
value2 = [{item.values()} for item in value2]
s = pd.Series([value1, value2], index=['value1', 'value2']) # <--- changes here
return s
mask = df['field'].isin(['target','target2'])
df.loc[mask, ['value1','value2']] = df[mask].apply(lambda row: f(row['field'], row['value1'], row['value2']), axis=1)
print(df)
field     value1             value2 value3
0   target   [{list}]  [{123}, {length}]  'abc'
1   random    [32.67]             [47.0]  'bcd'
2   target   [{word}]            [{105}]  'efg'
3  target2  [{(123)}]          [{(333)}]  'xyz'

{'source': 'StackExchangeDataset', 'subset': 'stackoverflow.com', 'metadata': {'AnswerCount': 1, 'CommentCount': 0, 'ContentLicense': 'CC BY-SA 4.0', 'CreationDate': '2022:00:06/12/22', 'FavoriteCount': None, 'PostTypeId': 1, 'Tags': '<python><pandas><dataframe>', 'Title': 'How do i change multiple columns of specific rows in pandas dataframe with custom functions?', 'ViewCount': 102, 'id': 72591729, 'user_AboutMe': None, 'user_AccountId': 21112098.0, 'user_DisplayName': 'Boon', 'user_Reputation': 45.0, 'user_Views': 4.0, 'user_id': 15519840.0}}