Read coordinates from many files and calculate polygon areas My goal with this snippet is to create an array of coordinates which in turn is a tuple of 68 elements, area and modified area array for all 10k elements and assign it to the df column.


Running the complete code results in 5.3, 4.8, 4.6, 4.5 seconds. 


#%%

arrayc = [] #array for array of coordinates
areaAr = [] #area type 1
modifiedarea = [] #area type 2
ts = time.time()

for i in range(10708): #number of files
        f = open(df["filepath"][i], "r") #df has column of filepaths 
        x,y = [], []
        for l in f:
            row = l.split()
            x.append(int(float(row[0]))) #68 pairs are of kind 3.82382323e+02 4.563524234e+02.
            y.append(int(float(row[1]))) #I am taking int rounded off to three digits.
        arrayc.append((x,y))
        f.close()
        #x= arrayc[i][0]
        #y = arrayc[i][1]
        areaAr.append(PolyArea(x[36:41],y[36:41]))
        distance = max(np.abs(x[36]-x[39]),np.abs(x[42]-x[45]))
        modifiedarea.append((PolyArea(x[36:41],y[36:41]))/distance)        

te = time.time()
print(-ts+te)
def PolyArea(x,y):
        return 0.5*np.abs(np.dot(x,np.roll(y,1))-np.dot(y,np.roll(x,1)))


How can I minimise the execution time ? 

Updates: 


File generator code:


import numpy as np 
filepath = []
root = '~/Desktop/test/'
for i in range(10):
    for j in range(68):
        numx = ( np.random.randint(100,200))
        numy = np.random.randint(100,200)
        f = open(root + str(i) + ".txt","a")
        f.write(str(numx) + " " + str(numy) + "\n")



The numbers 36 41 etc are the coordinates of the polygon of interest on the image. It is fixed that the polygon will always be marked by these coordinates.
I couldn't run your code, so I can't say how much faster the following will be.

My suggestion is to use NumPy's loadtxt function to get the array of the necessary coordinates. With this function, you can specify skiprows and max_rows parameters to get the necessary rows, 36-45. This should be more efficient than reading all the file in memory. 

Here:


areaAr.append(PolyArea(x[36:41],y[36:41])) 
distance = max(np.abs(x[36]-x[39]),np.abs(x[42]-x[45]))
modifiedarea.append((PolyArea(x[36:41],y[36:41]))/distance)        



you calculate PolyArea two times, but it is enough to calculate it only once and then reuse the result. 

The final code could lool like this:

for filepath in df['filepath'].iloc[:10708]:
    values = np.loadtxt(filepath, 
                        skiprows=35,
                        max_rows=10)
    x = values[:, 0]
    y = values[:, 1]
    area = PolyArea(x[:5], y[:5])
    areaAr.append(area)
    distance = max(np.abs(x[0] - x[3]), np.abs(x[6] - x[9]))
    modifiedarea.append(area / distance)