Find the 4 best parameters of a function to have the best cumulative sum I have a dataframe data with several columns and for each columns there are 10 years of daily data on average, something around 4000 rows then. The dataframe looks like this:
Date           A       B      C    ...
YYYY-MM-DD   123.4   56.7   89.0
YYYY-MM-DD   12.34   -5.6    NaN
...

Date is datetime64, every value is a float (positive and negative, they are real numbers), there could be NaN, there are no upside or downside boundaries.
I have this function :
def fct(data, name_a, name_b, param_a, param_b, param_c, param_d):
    
    df = data.loc[(data[name_a].notnull()) &amp; (data[name_b].notnull()),[name_a, name_b]]
    
    df_corr = pd.DataFrame()
    df_corr[name_a] = df[name_a]
    df_corr[name_b] = df[name_b].shift(param_a)
    print(df_corr.corr().iloc[0,1])
    
    df['pct'] = df[name_a].pct_change() 
    df['name_b_delta'] = df[name_b].shift(2+param_a)-df[name_b].shift(2+param_a+param_b)
    df['is_ok'] = df['name_b_delta'].apply(lambda x : np.sign(x+(-param_c)))
    
    if param_d:
        df['results'] = df['is_ok'] * (-1) * df['pct'] 
    else:
        df['results'] = df['is_ok'] * df['pct'] 
        
    df['cumul_results'] = df['results'].cumsum()             
                                                  
    return df

and currently, to optimize the cumul_results for differents pairs of name_a and name_b, I search for the best params using those nested loops :
name_a_list = ['a_dozen_of_elements_here']
name_b_list= ['25_elements_here']

df_line= []
df_index = []

for name_a in name_a_list :
    for name_b in name_b_list :
        for param_a in range(0,15):
            for param_b in range(1,15):
                for param_c in range (-2, 3, 1):
                    for param_d in [True, False]:
                        df = fct(data, name_a, name_b, param_a, param_b, param_c/1000, param_d):
                        df_index.append(f&quot;{name_a}-{name_b}&quot;)
                        df_line.append([df['cumul_results'].iloc[-1], param_a, param_b, param_c, param_d])

results = pd.DataFrame(index=df_index, data=df_line, columns=['final result', 'param_a', 'param_b', 'param_c', 'param_d'])

It's more than 840 000 calls of the function fct and takes around 4 hours. I was wondering then if I could increase the performance of this code to reduce time. Thanks !
Answer, part one
It looks like about half the loops can be avoided by doing the True and False cases at the same time.  These differ only by the multiplication by -1 at the end of fct.  That suggests that results of of a True invocation can be used to shortcut the computations of the immediately following invocation of fct.
Is there a reason to not cache the results of the several array accesses?  That is, for example, df[name_a], df[name_b], and others?
Can the apply and lambda be avoided?  This seems overly complex (meaning, slow) way to perform the computation.
I presume the print invocation is not there usually.  IO in the inner loop would slow things down considerably.
Answer, part 2
Sample code.  This runs for me, but I haven't verified that it produces exactly the same results.
I made two kinds of updates: Values which were shared across loops were lifted to the most common loop.  The True and False cases were collapsed into a single function call which returns the True and False results using multiple values.
Ignore my comment re: apply.
My gut tells me that at least the innermost shift invocation ought to be unnecessary, but I haven't figured how to write a correct apply that would remove it.  (I need the apply iteration index to be passed in as an extra parameter.)
I added a number of print statements for debugging which you will want to remove.
import pandas as pd
import numpy as np 

def fct_tf(data, name_a, name_b, param_a, param_b, param_c, df_common, df_shift):
    &quot;Compute results for two sets of parameters&quot;

    df_false = df_common.copy()

    # Lift one of the shifts up a loop.
    # Still working on removing the shifts altogether.
    # df_false['name_b_delta'] = df_false[name_b].shift(2+param_a)-df_false[name_b].shift(2+param_a+param_b)
    df_false['name_b_delta'] = df_shift - df_shift.shift(param_b)
    df_false['is_ok'] = df_false['name_b_delta'].apply(lambda x : np.sign(x+(-param_c)))

    df_false['results'] = df_false['is_ok'] * df_false['pct']
    df_false['cumul_results'] = df_false['results'].cumsum()

    df_true = df_false.copy()
    df_true['results'] = -1 * df_false['results']
    df_true['cumul_results'] = -1 * df_false['cumul_results']

    return df_true, df_false

def fct(data, name_a, name_b, param_a, param_b, param_c, param_d):
    &quot;Compute results for one set of parameters&quot;

    df = data.loc[(data[name_a].notnull()) &amp; (data[name_b].notnull()),[name_a, name_b]]

    df['pct'] = df[name_a].pct_change() 
    df['name_b_delta'] = df[name_b].shift(2+param_a)-df[name_b].shift(2+param_a+param_b)
    df['is_ok'] = df['name_b_delta'].apply(lambda x : np.sign(x+(-param_c)))

    if param_d:
        df['results'] = df['is_ok'] * (-1) * df['pct'] 
    else:
        df['results'] = df['is_ok'] * df['pct'] 

    df['cumul_results'] = df['results'].cumsum()             
                                                  
    return df

def compute(data, names_a, names_b):
    &quot;Compute results ...&quot;

    df_line= []
    df_index = []

    for name_a in names_a :
        print('name_a: ' + name_a)
        for name_b in names_b :
            print('name_b: ' + name_b)

            # 'df_common' will have just one column if the names are the same.
            # Then, an exception occurs when invoking 'pct_change()'.
            if ( name_a == name_b ):
                continue

            # This part of the results data is common across the parameters.
            df_common = data.loc[(data[name_a].notnull()) &amp; (data[name_b].notnull()),[name_a, name_b]]
            print('Common data:')
            print(df_common)

            # The percent change is also common.  Note that the values are not free from
            # a dependence on 'name_b', since the null check can change which rows are selected.
            # That means this value cannot be lifted to the 'name_a' loop.
            df_common['pct'] = df_common[name_a].pct_change()
            print(&quot;Common data[pct]:&quot;)
            print(df_common['pct'])

            for param_a in range(0,15):
                # Reduce the number of shifts
                # In theory, this shift should be avoidable, but I'm still figuring
                # how to remove it.
                df_shift = df_common[name_b].shift(2+param_a)

                for param_b in range(1,15):
                    for param_c in range (-2, 3, 1):
                        # The computations for 'False' and 'True' are the same, except
                        # for a change in sign.
                        # for param_d in [True, False]:
                            # df = fct(data, name_a, name_b, param_a, param_b, param_c/1000, param_d)
                            # df_index.append(f&quot;{name_a}-{name_b}&quot;)
                            # df_line.append([df['cumul_results'].iloc[-1], param_a, param_b, param_c, param_d])

                        df_true, df_false = fct_tf(data, name_a, name_b, param_a, param_b, param_c/1000, df_common, df_shift)

                        df_index.append(f&quot;{name_a}-{name_b}&quot;)
                        df_line.append([df_true['cumul_results'].iloc[-1], param_a, param_b, param_c, True])

                        df_index.append(f&quot;{name_a}-{name_b}&quot;)
                        df_line.append([df_false['cumul_results'].iloc[-1], param_a, param_b, param_c, False])

    results = pd.DataFrame(index=df_index, data=df_line, columns=['final result', 'param_a', 'param_b', 'param_c', 'param_d'])

    return results