Q: Code taking too much time for normalisation and Apply function in Pandas this is just sample data with my code. i want to normalization the total column in other column. currently i have around 2000 groups for that its taking 15 mins for normalisation and fgroup.
what are the ways to reduce the time.
import pandas as pd
import numpy as np

np.random.seed(1234)
n = 1500000

df = pd.DataFrame()
df['group'] = np.random.randint(1700, size=n)
df['ID'] = np.random.randint(5, size=n)
df['Total'] = np.random.randint(400, size=n)
df['Normalized_total'] = df.groupby('group')['Total'].apply(lambda x: (x-x.min())/(x.max()- x.min()))

def norm_group(df):
    if df['Normalized_total'] > 0.70 and df['group'] > 100 and df['ID'] > 3:
        return 3
    elif df['Normalized_total'] > 0.5 and df['group'] < 100 and df['ID'] < 3:
        return 2
    else:
        return 1

df['fgroup'] = df.apply(norm_group, axis=1)

Thanks

A: You can using transform and define your own function 
%timeit df['Normalized_total'] = df.groupby('group')['Total'].apply(lambda x: (x-x.min())/(x.max()- x.min()))
1 loop, best of 3: 508 ms per loop

# below is my solution
def myfunc():
    g=df.groupby('group')['Total']
    return df['Total']-g.transform('min')/g.transform(np.ptp)
%timeit myfunc()
1 loop, best of 3: 398 ms per loop

