Pandas add calculated row for every row in a dataframe
iterrows
Of all the ways to iterate over a pandas DataFrame, iterrows is the worst. This creates a new series for each row. this series also has a single dtype, so it gets upcast to the least general type needed. This can lead to unexpected loss of information (large ints converted to floats), or loss in performance (object dtype).
Slightly better is itertuples
append
Appending row per row can be very slow (link1link2)
Better would be to assembly them in a list, and make a new DataFrame in 1 go.
vectorise
One easy change you can make is not iterating over the database in 'Python' space, but using boolean indexing
x = df[&quot;variable&quot;] == &quot;x&quot;
y = df[&quot;variable&quot;] == &quot;y&quot;

df2 = df.copy()
df2.loc[x, &quot;value&quot;] = df.loc[x, &quot;value&quot;] * a + b
df2.loc[y, &quot;value&quot;] = df.loc[y, &quot;value&quot;] * c + d
df2.loc[x, &quot;variable&quot;] = &quot;x1&quot;
df2.loc[y, &quot;variable&quot;] = &quot;y1&quot;

wide vs long
This can be made a lot easier by reforming your dataframe by making it a bit wider:
df_reformed = (
    df.set_index([&quot;id&quot;, &quot;variable&quot;]).unstack(&quot;variable&quot;).droplevel(0, axis=1)
)


variable  x   y
id        
1     5   5
2     7   7


Then you can calculate x1 and y1 vectorised:
df_reformed.assign(
    x1=df_reformed[&quot;x&quot;] * a + b, 
    y1=df_reformed[&quot;y&quot;] * c + d
)

and then convert this back to the long format:
result = (
    df_reformed.assign(
        x1=df_reformed[&quot;x&quot;] * a + b, 
        y1=df_reformed[&quot;y&quot;] * c + d
    )
    .stack()
    .rename(&quot;value&quot;)
    .sort_index()
    .reset_index()
)