How to add rows to pandas dataframe with reasonable performance
Where you use append you end up copying the dataframe which is inefficient. Try this whole thing again but avoiding this line: 
df = df.append(df.iloc[-1]) 
You could do something like this to copy the last row to a new row (only do this if the last row contains information that you want in the new row): 
df.iloc[...calculate the next available index...] = df.iloc[-1]
Then edit the last row accordingly as you have done 
df.iloc[-1, df.columns.get_loc('column_name')] = some_extracted_value
I have an empty data frame with about 120 columns, I want to fill it using data I have in a file.
I'm iterating over a file that has about 1.8 million lines.
(The lines are unstructured, I can't load them to a dataframe directly)
For each line in the file I do the following:

Extract the data I need from the current line
Copy the last row in the data frame and append it to the end df = df.append(df.iloc[-1]). The copy is critical, most of the data in the previous row won't be changed.
Change several values in the last row according to the data I've extracted df.iloc[-1, df.columns.get_loc('column_name')] = some_extracted_value

This is very slow, I assume the fault is in the append.
What is the correct approach to speed things up ? preallocate the dataframe ? 
EDIT:
After reading the answers I did the following:

I preallocated the dataframe (saved like 10% of the time)
I replaced this : df = df.append(df.iloc[-1]) with this : df.iloc[i] = df.iloc[i-1] (i is the current iteration in the loop).(save like 10% of the time).
Did profiling, even though I removed the append the main issue is copying the previous line, meaning : df.iloc[i] = df.iloc[i-1] takes about 95% of the time.