CSV, keep file open between each run/measurement
Question: I have a question regarding writing CSV-files. I have an instrument where I am
going to read a value each minute and write the value to a CSV file together
with a timestamp.

I have written a simple code which works, which is the first one below. But my
friend came up with a question: "Why are you opening and closing the file for
each run." I couldn't answer his question since I'm a newbie when it comes to
both programming and Python. Nevertheless, I tried to rewrite the code, so it
became like solution number 2 below and then asked my friend whether that was
better, but he couldn't answer me. So I hope you could help me out, what is
the difference between #1 and #2, both gives the same result when it comes to
reading the file afterwards.

Is it right that #1 will open and close the file for each run in the while
loop, and #2 will keep the file open and close it after the last run?

This is just a simple example, in reality, I am going to expand the code so it
will do a measurement every minute, run for 3-4 days and I am going to log
multiple voltages and up to 8 temperatures. So, in the end, the file can
become quite large and opening and closing the file can become a bit resource
consuming.

> Code #1:
    
    
    import csv
    import datetime
    import time
    
    no_meas = 5 #number of measurements
    cur_meas = 1    # current measurement number
    
    
    while cur_meas <= no_meas:
        cur_time = datetime.datetime.strftime(datetime.datetime.now(), '%H:%M:%S')
        with open('test.csv', 'a', newline='') as fp:
            a = csv.writer(fp, delimiter=',')
            data = [[cur_time, 'test-text']]
            a.writerows(data)
        cur_meas += 1
        time.sleep(60)
    

> Code #2
    
    
    import csv
    import datetime
    import time
    
    no_meas = 5 #number of measurements
    cur_meas = 1    # current measurement number
    
    with open('test.csv', 'a', newline='') as fp:
        while cur_meas <= no_of_meas:
            cur_time = datetime.datetime.strftime(datetime.datetime.now(), '%H:%M:%S')
            a = csv.writer(fp, delimiter=csv_delimiter)
            data = [[cur_time, 'test-text']]
            a.writerows(data)
            cur_meas += 1
            time.sleep(60)
    


Answer: If the the frequency of your writes to the file is high you may want to avoid
the open/close overhead. Nevertheless, from what you describe it doesn't seem
like an issue (1 open/close per minute for writing a couple of dozens of
measurements). Even after a few days your file should not exceed the 1-3MB
size.

Since you're planning to let it run for a few days, you should handle a
scenario of failure without losing the gathered data. In the first case you
won't lose any data. In the second case, I'd recommend adding a flush after
each write to make sure the data is written to disk:

    
    
    a.writerows(data)
    fp.flush()
    os.fsync(fp.fileno())
    

